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Preface 


The  purpose  of  this  study  was  to  evaluate  the  methods 
and  sources  of  microcomputer  training  currently  used  in  the 
Air  Force.  I  hope  my  findings  and  further  studies  can 
provide  government  managers  with  the  information  they  need 
to  choose  the  most  cost-effective  training  for  employees. 

I  surveyed  a  sample  of  151  people  from  the  19,000  who 
work  in  AFLC  and  ASD  on  Wright-Patterson  AFB  to  compare 
levels  of  training  effectiveness,  employee  computer 
competence,  and  training  value  to  government  employees  which 
result  from  three  methods  of  PC  training:  classroom,  video- 
based,  and  computer-based  (CBT) .  I  wanted  to  evaluate  three 
sources  of  such  training:  in-house,  contractor,  and  private 
(e.g.  university,  college) .  I  believe  this  study  can  be 
used  as  the  framework  for  additional  studies  which  could 
provide  better  evidence  for  which  training  methods  should  be 
used  and  which  should  be  avoided  in  the  future. 

Thank-you  to  the  people  who  assisted  me  in  this 
research:  The  folks  at  HQ,  Air  Force  Logistics  Command; 

Aeronautical  Systems  Division;  and  the  USAF  Small  Computer 
Office  Automation  Service  Center.  A  big  thanks  to  Professor 
Jeff  Daneman,  my  thesis  advisor.  I  also  want  to  thank  my 
friends  Greg  Allen  and  Cheryl  Duckett  for  their  support.  I 
especially  want  to  thank  my  wife,  Judy  for  her  patience, 
understanding,  encouragement,  and  love. 

Mike  Kondas 
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Abstract 

The  purpose  of  this  study  was  to  evaluate  the  methods 
and  sources  of  microcomputer  training  currently  used  in  the 
Air  Force  in  terms  of  their  overall  value  to  the  job, 
training  effectiveness,  and  level  of  competency  achieved 
upon  completion.  The  three  methods  studied  were  classroom 
instruction,  video-based  training,  and  computer-based 
training  (CBT) .  The  three  sources  studied  were  in-house 
training  functions,  government-contracted  training,  and 
private  (i.e.,  colleges  and  universities). 

One  hundred  and  fifty-one  government  employees  rated 
each  method  and  source  of  PC  training  they  had  experienced. 
These  ratings  were  used  to  produce  a  ranking  which  can  be 
found  in  chapter  V. 

Classroom  training  was  found  to  be  the  most  often  used 
and  the  most  popular  method  of  PC  training,  regardless  of 
source.  The  preference  of  delivery  method  was  classroom, 
CBT,  then  video-based  training,  regardless  of  source. 

A  cost-benefit  analysis  was  performed  to  compare  the 
two  most  often  used  sources  of  classroom  training:  in-house 
and  government-contractor.  Statistical  results  indicated 
that  the  two  sources  are  equal  in  terms  of  the  benefits 
stated  above.  Cost-per-training  hour  was  slightly  lower  for 
vendor-provided  training  than  for  in-house  training. 
Resources  required  to  establish  and  maintain  in-house 
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training  functions  were  substantially  greater  than  those 
required  to  support  vendor-provided  training  programs. 

The  study  found  that  college  or  university  PC  training 
was  rated  equally  with  in-house  and  commercial  sources 
regardless  of  method. 

One  particular  recommendation,  in  light  of  the 
challenge  presented  to  managers  for  improving  microcomputer 
training  in  the  workplace,  is  to  investigate  the  possibility 
of  training  government  employees  by  contract  with  local 
educational  institutions.  Potential  benefits  include  better 
trained  and  educated  employees,  higher  morale,  and  savings 
of  operations  and  maintenance  funds.  Savings  of  O&M  funds 
are  possible  in  two  ways.  The  first  is  that  employees  could 
use  tuition  assistance  funding  since  such  training  also 
meets  individual  educational  needs.  The  second  is  that  such 
training,  if  atteiided  in  off-duty  hours,  could  mean  higher 
levels  of  productivity.  Again,  these  possibilities, 
although  promising,  require  further  investigation. 
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A  COMPARISON  OF  MICROCOMPUTER  TRAINING 


METHODS  AND  SOURCES 


I.  Introduction 


General  Issue 

The  introduction  of  microcomputers  (also  known  as 
personal  computers,  desktops,  PCs,  and  micros)  into  the  Air 
Force  work  environment  created  an  exceptional  potential  for 
increases  in  productivity.  But  the  full  productivity 
benefits  of  microcomputers  have  yet  to  be  realized  in  the 
Air  Force  due  to  lack  of  accurate  information  for  decision¬ 
making  about  which  method  of  training  to  choose  for 
employees  who  are  end-users  of  microcomputers  (19;  28;  10). 

U.  S.  Air  Force  investment  in  microcomputers  will 
represent  a  large  portion  of  the  nearly  400  billion  dollars 
projected  to  be  spent  on  microcomputers  in  America  by  1990 
(8:17).  The  plan  for  a  computer  on  every  USAF  desktop  is 
approaching  fruition  (10).  In  light  of  this  success 
however,  managers  are  faced  with  the  challenge  of  ensuring 
that  every  PC  is  used  to  the  maximum  extent  possible  to 
increase  productivity  rather  than  collect  dust.  To  do  so, 
they  must  find  the  most  effective  training  for  their 
employees  at  the  lowest  possible  cost. 
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A  review  of  current  literature  revealed  that  no  studies 
have  been  published  within  the  Department  of  Defense  or  USAF 
which  address  the  impacts  in  the  workplace  of  various 
microcomputer  training  methods  .  Succinctly,  no  studies 
were  found  which  address  microcomputer  training 
effectiveness,  the  resulting  employee  computer  competency 
level,  or  the  overall  value  of  such  training  to  the  employee 
within  the  DOD  or  USAF.  Furthermore,  little  data  exists  to 
identify  which  methods  make  employees  more  productive,  or 
equally  important,  which  methods  are  the  most  cost- 
effective.  If  more  complete  and  accurate  information  were 
available,  decision  makers  could  be  more  successful  at 
choosing  more  cost-effective  methods  microcomputer  training 
methods . 

Background 

The  Air  Force,  it  its  initial  efforts  to  establish 
microcomputer  training,  followed  the  traditional  pattern  for 
meeting  military  training  needs.  Air  Training  Command  (ATC) 
established  the  Small  Computer  Training  branch  at  Keesler 
Air  Force  Base,  Mississippi  in  March  1984  (19) .  The  Air 
Force  Communications  Command  (AFCC)  introduced  initiatives 
to  develop  small  computer  support  focal  points  at  bases 
worldwide  during  the  mid-1980s  (28).  These  focal  points  are 
now  called  Small  Computer  Customer  Centers.  Success  ol 
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these  support  efforts,  which  is  discussed  in  more  detail  in 
chapter  II,  has  been  mixed.  (19;  28) 

Part  of  the  problem  stems  from  the  fact  that  management 
had  failed  to  recognize  the  need  to  address  end-user 
training  in  the  early  stages  of  microcomputer  growth  in  the 
DOD  (18:2).  Arthur  Young  &  Company  performed  a  study  for 
the  Office  of  the  Assistant  Secretary  of  Defense 
(Comptroller)  as  part  of  a  project  designed  to  identify 
management  issues  and  concerns  and  recommend  alternatives 
for  a  comprehensive  approach  to  managing  the  growth  in 
computer  technology.  The  1983  report,  containing  the 
findings  for  that  portion  of  a  project  which  dealt  with  user 
support,  mentioned  training  once  -  one  word,  one  sentence. 

(6) 

Some  efforts  were  made  to  identify  the  need  for  PC 
training,  but  these  efforts  were  limited  and  have  proven  to 
be  less  than  successful.  (28;-  10;  29)  The  1984  Air  Force 
Management  Analysis  Group  on  Data  Systems  Management  and 
Manpower  Impacts  recommended  that,  "a  major  campaign  should 
be  conducted  to  improve  the  computer  literacy  of  Air  Force 
personnel"  (18:17-8).  One  reason  that  such  a  campaign  has 
not  been  successfully  undertaken  is  the  perennial  problem  of 
a  lack  of  proper  information  for  decision  making  -  the 
subject  of  this  thesis  (10). 
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Specific  Problem 

The  specific  problem  which  this  thesis  addresses  is 
that  faced  by  Air  Force  managers  of  microcomputers  and  the 
employees  who  use  them: 

Which  microcomputer  training  methods  result  in  the 
highest  levels  of  training  effectiveness,  computer 
competency,  and  value  to  the  employee  per  dollar  spent? 

Particular  Approach 

To  find  a  solution  to  the  specific  problem,  the 
particular  approach  used  in  this  thesis  included  a  review  of 
the  progress  made  thus  far  in  meeting  microcomputer  training 
needs  in  the  Air  Force;  a  review  of  currently  available 
training  methods;  interviews  of  selected  Air  Force  personnel 
involved  in  decision-  and  policy-making  for  microcomputer 
training;  a  survey  of  USAF  employees  who  have  received 
microcomputer  training;  and  the  application  of  a  cost- 
benefit  analysis  of  the  two  most  prevalent  microcomputer 
training  methods  currently  used  by  the  Air  Force, 

Existing  literature  about  various  microcomputer 
training  methods  used  by  other  modern  organizations  was 
reviewed.  Additionally,  selected  Air  Force  personnel 
involved  in  microcomputer  training  issues  and  policy  making 
were  interviewed  to  determine  what  the  Air  Force  has 
recently  done,  or  is  planning  to  do  to  meet  the  changing 
needs  in  microcomputer  training.  These  individuals  were 
assigned  to  the  staffs  of  Air  Training  Command,  Air  Force 
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Communications  Command,  Air  Force  Logistics  Command,  the 
Department  of  Defense  Small  Computer  Office  Automation 
Service  Activity  at  Gunter  AFB,  Alabama,  and  the 
Aeronautical  Systems  Division. 

A  literature  review  indicated  that  many  private 
sector  organizations  employ  cost-benefit  analyses  to  decide 
which  training  sources  and  methods  to  use  for  their 
employees.  This  thesis  includes  such  an  analysis  of  two 
particular  sources  of  the  classroom  method  of  training  used 
by  the  Air  Force,  in-house  and  commercial.  Two  other 
methods  of  PC  training  -  computer-based  and  video-based  - 
and  one  other  source  of  training  -  colleges  and  universities 
-  are  also  evaluated,  but  only  with  regards  to  benefits 
since  cost  data  was  not  available  for  this  research. 

To  determine  the  benefits  of  each  method,  a 
questionnaire-survey  was  administered  to  government 
employees  who  received  one  or  more  types  of  microcomputer 
training  in  the  past  three  years.  This  survey  was  used  to 
collect  data  on  levels  of  microcomputer  training 
effectiveness,  employee  computer  competency,  and  overall 
value  to  government  employees. 

Scope 

Training  Sources .  There  are  a  number  of  sources  form 
which  the  Air  Force  can  currently  obtain  PC  training.  Time 
constraints  and  lack  of  objective  data  for  all  sources  and 
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methods  of  training  consequently  limited  this  study  to  threeo 
sources:  in-house,  vendor-provided,  and  college  or 
university,  referred  to  as  private  training.  In-house 
training  is  defined  as  that  training  given  to  government 
employees  using  government  facilities  and  instructors. 
Vendor-provided  training  is  defined  as  that  training  given 
to  government  employees  using  non-government  facilities  and, 
or  instructors. 

The  cost-benefit  analysis  herein  addresses  only  in- 
house  and  commercial  sources.  However,  some  preliminary 
findings  concerning  private  sources  of  PC  training  are  also 
presented.  The  primary  method  used  as  a  basis  for 
comparison  of  in-house  and  commercial  sources  was  classroom 
instruction.  This  particular  method  was  chosen  over  others 
since  it  represents  the  largest  investment  of  training 
dollars  and  hours  (10)  and  substantial  data  sources  for  each 
source  were  readily  available  within  the  same  geographical 
region,  i.e.,  the  area  surrounding  Wright-Patterson  Air 
Force  Base,  Ohio. 

Models  Used.  For  purposes  of  this  thesis,  two  real- 
life  models  were  used  for  data  collection  and  research.  The 
government-operated  training  facility  located  in  building 
110  on  Wright-Patterson  AFB,  Ohio  serves  as  the  model  for 
in-house  training  provided  to  government  employees  working 
primarily  in  Air  Force  Logistics  Command  organizations.  The 
real-life  model  for  vendor-provided  training  is  the 
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Aeronautical  Systems  Division  (ASD)  of  Air  Force  Logistics 
Command.  ASD  receives  over  99%  of  its  training  from  public 
contractors  located  in  the  area  surrounding  Wright-Patterson 
AFB  (17) .  These  companies  include  J  and  K  Associates  of 
Dayton,  VALCOM  Training  Center  also  of  Dayton,  and  Logistics 
Systems  Architects  (LSA) ,  located  in  Fairborn,  Ohio.  J  and 
K  Associates  and  VALCOM  provide  almost  fifty  percent  each  of 
the  training  provided  to  ASD  (17) .  LSA,  which  currently 
handles  only  a  small  percentage  of  training  for  ASD,  also 
has  offices  near  the  Air  Logistics  Centers  at  Midwest  City, 
Oklahoma,  Sacramento,  California,  and  Warner  Robins,  Georgia 
(7)  . 

Costs .  The  author  has  made  every  attempt  to  capture 
all  costs  associated  with  each  of  the  two  types  of  training. 
In  cases  where  costs  would  be  the  same  for  either  source, 
e.g.,  employee  time  away  from  normal  duties,  costs  are  not 
reflected . 

Benefits .  The  level  and  types  of  benefits  which  accrue 
from  the  many  types  of  microcomputer  training  were  numerous. 
Three  particular  types  of  benefits  are  frequently  used  as 
the  basis  for  private  sector  source/method  selection,  and 
were  chosen  as  most  important  from  an  Air  Force  manager's 
perspective.  The  first  of  the  three  is  microcomputer 
training  effectiveness  which  was  measured  by  how  much 
material  a  student  can  learn  in  one  hour  of  training.  A 
ratio  of  the  level  of  material  learned  to  the  cost  of  one 
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hour  of  such  training  can  be  computed  which  facilitates  the 
comparison  of  one  method  to  another-  The  second  benefit 
used  is  employee  computer  competence  -  as  measured  by  the 
level  of  computer  competence  achieved  as  a  result  of  the 
type  of  training  taken  rather  than  the  course  material. 

Here,  a  ratio,  based  on  a  percentage  of  material  mastered, 
can  be  used  to  compare  methods.  Finally,  the  last  measure 
surveyed  is  the  level  of  overall  value  to  the  government 
employee.  This  particular  benefit  was  chosen  since, 
frequently,  training  methods  do  not  focus  on  the  training 
needed  in  the  employee's  duties,  but  rather  provide  a 
generic  overall  exposure  of  material  (14:25).  In  such 
cases,  training  costs  could  be  considered  wasted  since 
learned  material  can  be  forgotten  as  a  result  of  non-use 
(14:25;  20:40). 

Summary 

This  chapter  has  introduced  the  general  issue, 
microcomputer  training  methods,  and  provided  a  brief 
background  which  highlights  an  overall  problem  in  the 
inability  of  current  USAF  computer  training  programs  to  meet 
all  current  training  requirements.  In  light  of  this 
shortcoming,  the  specific  problem  to  be  addressed  by  this 
thesis  is  which  sources  are  most  cost-effective  when 
compared  in  terms  of  three  benefits  achieved  through 
classroom  instruction.  The  three  benefits  are  training 
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effectiveness,  computer  competency,  and  overall  value  to  the 
government  employee  and  will  serve  as  a  basis  for  comparison 
for  the  two  most  prevalent  sources  currently  used  in  the  Air 
Force  -  In-house  and  Vendor-provided. 
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II .  Literature  Review 


Introduction 

This  chapter  will  address  several  topics  associated 
with  the  management  of  microcomputers  in  modern 
organizations  while  focusing  on  the  progress  made  thus  far 
in  the  Air  Force.  Using  a  model  for  the  life  cycle  of 
computers  in  organizations,  a  discussion  of  the  importance 
of  planning  for  microcomputers  is  presented.  A  primary 
element  addressed  in  such  planning  -  training  -  is  covered 
next.  The  remainder  of  the  chapter  is  devoted  to  a 
discussion  of  each  of  the  more  popular  methods  of 
microcomputer  training  currently  available  with  a  focus  on 
in-house  and  vendor-provided  classroom  training. 

Managing  Computers  in  Modern  Organizations 

Successful  management  of  automation  in  today's 
organizations  requires  effective  integration  of  computers 
into  the  workplace.  (23:324;  25:121).  One  survey  of 
management  information  system  experts  ranked  this 
management  challenge  as  their  "second  most  important ...  for 
the  1980's"  (15:353).  Two  major  management  responsibilities 
in  integration  are  planning  and  training  (25:121). 

The  Importance  of  Planning 

Planning  for  microcomputers  is  a  necessary  management 
process.  In  his  book  entitled,  "Microcomputers  in  Large 
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Organizations,”  Thomas  Madron  explains  the  planning  process; 

The  planning  process  implies  that  several  things 
have  happened  in  the  organization.  First,  planning 
necessitates  a  clear  understanding  of  the  problems 
involved..  Second,  it  requires  that  some  institu¬ 
tional  policies  and  goals  be  established....  And 
third,  it  involves  the  elaboration  of  appropriate 
procedures  for  implementing  the  policies  and 
solving  the  problems.  (25:116) 

Madron  states,  "proper  planning. .. is  essential  for  micros  to 

be  used  well"  {25;vii) . 

Managers  are  beginning  to  recognize  the  significance  of 
proper  strategic  planning.  In  a  1985  survey  of  management 
information  system  managers.  Dr.  Curt  Hartog,  Assistant 
Director  of  the  Center  for  the  Study  of  Data  Processing  at 
Washington  University,  St.  Louis,  Missouri,  found  that  long 
range  planning  was  the  highest  rated  issue  of  concern. 
(15:352) 

The  Nolan  Stage  Model .  A  model  can  be  used  to 
illuminate  the  significance  of  planning  during  the  life 
cycle  of  computers  in  an  organization.  The  Nolan  Stage 
Model,  developed  by  Cyrus  F.  Gibson  and  Richard  L.  Nolan, 
identifies  four  distinct  stages  of  the  evolution  of 
information  system  activities  and  their  related  problems 
within  organizations.  The  four  stages  of  this  model  are: 
Initiation,  Expansion,  Formalization,  and  Maturity.  (9) 

Stage  1,  Initiation,  involves  the  initial  acquisition 
of  computers.  In  his  thesis  on  "End  User  Software  Develop¬ 
ment  for  Transportation  Applications,"  Caver  describes  the 
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typical  situation  faced  by  modern  organizations  when  imple¬ 
menting  microcomputers: 

At  this  stage  long  term  implications  are  rarely 
considered  by  management,  and  there  is  little 
strategic  planning  for  other  impacts  of  the 
computer  on  personnel,  or  the  organization.  (2:14) 

Stage  2,  Expansion,  is  described  by  Caver  as  the 

"...rapid  and  uncontrolled  growth  in  the  use  and  acquisition 

of  computer  systems"  (2:14).  This  chaotic  growth,  combined 

with  a  lack  of  adequate  management  involvement,  results  in 

rapid  cost  increases  which  are  not  planned  for  before 

computers  are  acquired.  (2:14;  9:80-83;  16:10-11) 

Stage  3,  Formalization,  begins  with  management’s 

attempts  to  gain  some  control  over  the  disorder  of  the 

Expansion  stage.  Control  becomes  centralized  as  formal 

planning,  standardization,  and  integration  begin.  (2:14; 

9:83-86;  16:11-12) 

In  Stage  4,  Maturity,  management  regains  control. 
Computer  policy  becomes  integrated  with  overall  corporate 
policies  and  computers  are  ultimately  managed  as  effectively 
as  all  other  resources.  (2:15;  9:86-88;  16:12) 

Experts  agree  that  most  companies  are  still  in  the 
Initiation  or  Expansion  stages  of  this  model.  As  such, 
planning  in  these  organizations  is  lacking.  (9;  16) 

Planning  In  the  Air  Force .  Caver  implies  that  the  Air 
Force  is  in  stage  2  or  3,  along  with  many  of  its  civilian 
counterparts,  by  stating,  "...the  Air  Force  has  focused  the 
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majority  of  its  attention  on  obtaining  microcomputer 


hardware  and  software...”  (2:23).  There  is  evidence  that 

the  Air  Force  does  have  some  work  to  do  (to  reach  Stage  4, 

Maturity)  in  managing  microcomputers  as  effectively  as  all 

other  resources  since  training  needs  are  not  being  met 

entirely.  As  mentioned  earlier  in  chapter  I,  the  1984  Air 

Force  Management  Analysis  Group  on  Data  Systems  Management 

and  Manpower  Impacts  recommended  that,  ”a  major  campaign 

should  be  conducted  to  improve  the  computer  literacy  of  Air 

Force  personnel."  (18:17-8)  The  efforts  that  resulted  from 

this  recommendation  are  discussed  later  in  this  chapter 

under  Training  in  the  Air  Force. 

Planning  Outside  the  Air  Force .  The  Air  Force’s  lack 

of  successful  planning  is  not  atypical.  Management  experts 

at  Case  Western  Reserve  University  in  Cleveland,  Ohio, 

performed  a  survey  of  173  Forbes  500  companies  which  use 

personal  computers  and  found  that; 

. . .many  problems  appear  to  be  related  to  the  lack 
of  clear  policies  on  the  part  of  top  management... 
greater  attention  is  needed  to. .. techniques  of 
strategic  planning  for  personal  computing.  (12:184) 

The  researchers  indicated,  "the  most  critical  problem... is 

lack  of ...  long-term  perspective..."  (12:182).  The  lack  of 

adequate  planning  in  the  management  of  computers  can  lead  to 

a  multitude  of  other  problems;  one  of  the  more  serious 

problems  is  the  lack  of  proper  training.  (12;  2;  25;  24;  23) 
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The  Need  for  Training 

While  managers  find  that  lack  of  planning  is  the  most 
serious  computer  management  problem  today,  employees 
indicate  that  lack  of  training  is  a  much  more  significant 
problem  (23:324).  A  1985  study  about  sources  of  assistance 
for  end  users  of  microcomputers  in  the  private  sector 
revealed  basic  trends  occurring  among  employees  using 
microcomputers.  The  study  revealed  that  the  lack  of  formal 
training  caused  less  than  effective  use  of  computers.  The 
study  also  found  that  the  most  popular  source  for  computer 
assistance  when  formal  training  wasn't  available  was 
coworkers.  The  next  best  source  of  information  was  the  in- 
house  information  systems  staff;  use  of  this  source, 
however,  was  less  than  expected.  (23:324)  The  next  section 
will  demonstrate  how  similar  trends  have  also  occurred  in 
the  USAF. 

Training  in  the  Air  Force.  Air  Training  Command 
established  the  Small  Computer  Training  branch  at  Keesler 
Air  Force  Base,  Mississippi  in  March  1984  as  a  first  step  to 
provide  microcomputer  training  to  end-users  in  the  Air 
Force.  The  branch  offers  in-house  training  and  supports  a 
mobile  team  which  travels  to  Air  Force  bases  throughout  the 
world.  Nine  instructors  must  staff  the  resident  course  at 
Keesler  and  travel  up  to  350  days  a  year  to  train  as  many  as 
9500  students  per  year.  (19) 
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The  team  chief,  MSgt  Justin  Johns,  indicates  that  the 
demand  for  training  is  growing  but  not  being  met  through  Air 
Force  resources: 

Up  to  35,000  people  need  training  right  now.  Some 
bases  are  paying  contractors  up  to  $1000  a  day 
because  they  can't  wait  for  us.  We  just  don't 
have  the  money  or  manpower.  (19) 

Other  organizations  have  sought  to  fill  the  training 
gaps  that  ATC  resources  cannot  meet.  Air  Force 
Communications  Command  introduced  small  computer  support 
focal  points  at  bases  worldwide  during  the  mid-1980s,  now 
called  Small  Computer  Customer  Centers.  But  these  centers 
are  limited  in  the  level  of  support  they  can  offer.  (19;  28) 
Essentially,  these  centers  provide  a  microcomputer 
consulting  service  upon  request.  One  reason  for  their 
limited  success  is  that  users  don't  necessarily  possess 
enough  basic  )cnowledge  of  computers  to  know  what  to  types  of 
service  or  training  to  request.  According  to  Captain  Albert 
Olague,  Chief  of  Computer  Requirements  for  AFCC,  "Many  users 
don't  have  sufficient  computer  knowledge  to  know  what 
questions  to  ask:  that's  definitely  a  training  problem" 

(28)  . 

Captain  Olague  and  MSgt  Johns  both  agree  that  these 
problems  have  reduced  the  level  of  productivity  expected 
from  microcomputers  in  the  Air  Force  (19;  28).  MSgt  Johns 
has  traveled  to  Air  Force  bases  worldwide  for  over  four 
years  to  train  microcomputer  users.  He  estimates  that  as 


many  as  one  in  seven  micros  are  not  used  due  to  the  lack  of 
proper  training  (19) .  As  micros  appear  daily  on  Air  Force 
bases  around  the  world,  the  demand  for  training  is 
frequently  filled  by  Air  Force  employees  working  with  each 
other  or  forming  base  level  computer  groups.  This  form  of 
training  usually  takes  more  time,  is  less  effective  than 
formal  training,  and  in  some  cases  results  in  actual 
productivity  decreases.  (19;  28). 

These  problems  apparently  did  not  go- unnoticed.  As 
mentioned  earlier  in  chapter  I,  the  1984  Air  Force 
Management  Analysis  Group  on  Data  Systems  Management  and 
Manpower  Impacts  recommended  that,  "a  major  campaign  should 
be  conducted  to  improve  the  computer  literacy  of  Air  Force 
personnel."  (18:17-8) 

Not  until  recently  has  such  a  campaign  been  pursued  on 
any  large  scale.  In  1988,  the  staffs  of  the  Air  Force 
Communications  Command  at  Scott  AFB,  Illinois  and  the  Air 
Force  Small  Computer  Office  Automation  Service  at  Gunter 
AFB,  Alabama  (AFSCOAS)  began  efforts  to  develop  a  means  of 
ensuring  that  microcomputer  training  would  be  available  when 
and  where  needed  throughout  the  USAF.  Since  current 
hardware  and  software  acquisition  contracts  frequently  do 
not  include  provisions  for  other  than  limited  orientation 
training,  AFCC  began  considering  a  "Training  Only"  contract; 
negotiated  as  a  tri-service  contract  to  realize  greater 
volume  savings.  Before  any  significant  results  were 
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achieved,  the  Air  Staff  turned  over  the  major  portion  of 
responsibility  for  small  computer  training  issues  to  AFSCOAS 
during  the  winter  of  1988.  (28;  10) 

Coincidentally,  the  AFSCOAS  was  in  the  process  of 
awarding  a  new  multi-user  system  contract  and  writing  a  new 
Desktop  contract.  Both  of  these  contracts  cover  a  variety 
of  small  computers,  associated  software,  system  support,  and 
for  the  first  time,  more  formalized  training  (28) .  Both  the 
AFCC  and  AFSCOAS  staffs  expressed  concern  over  the  fact  that 
users  who  were  not  covered  by  these  two  new  contracts  would 
not  be  guaranteed  training  when  and  where  it  was  needed  (28; 
10)  . 

Since  such  gaps  still  exist  in  meeting  the  training 
needs  of  end-users,  bases  are  looking  for  new  ways  to  meet 
the  expanding  PC  training  requirements.  Recent  efforts  at 
more  than  a  few  bases  have  resulted  in  some  rather 
innovative  approaches  to  training  employees  who  use  PCs. 

For  example,  according  to  the  AFSCOAS,  a  number  of  bases  are 
working  with  local  community  colleges  to  provide  training  to 
base  employees.  Among  the  results  are  lower  costs,  improved 
community  relations,  and  very  satisfied  employees  (who 
receive  quality  training  and  college  credit) .  (10) 

A  number  of  bases  are  filling  the  training  gap  by 
establishing  their  own  in-house  training  facilities  or  by 
contracting  with  local  vendors  for  PC  training.  Examples  of 
each  of  these  training  methods  are  the  programs  designed  to 
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meet  the  training  needs  of  micro  users  at  Wright-Patterson 
AFB  Ohio. 

The  Air  Force  Logistics  Command  (AFLC)  has  created  its 
own  in-house  formal  training  facility.  The  facility, 
located  in  building  110  in  Area  A  on  Wright-Patterson  AFB, 
is  officially  referred  to  as  the  Communications-Computer 
Systems  Training  Facility  for  Headquarters,  Air  Force 
Logistics  Command  (29) . 

The  Aeronautical  Systems  Division  (ASD)  of  Air  Force 
Systems  Command  (AFSC)  satisfies  over  99  percent  of  its 
computer  training  needs  by  contracting  with  local  companies 
which  specialize  in  such  training.  J  and  K  Associates  of 
Dayton  provides  nearly  50  percent  of  the  training  given  to 
ASD  employees.  VALCON  Training  Center  of  Dayton  also 
handles  almost  another  50  percent  of  ASD  training 
requirements.  Logistics  Systems  Architects,  located  in 
Fairborn,  Ohio,  currently  provides  the  remaining  portion  of 
ASD  microcomputer  training.  Approximately  35  to  40  percent 
of  the  training  courses  are  taught  at  the  vendors' 
facilities  located  off-base.  The  remainder  of  the  class  are 
taught  in  classrooms  located  within  ASD  facilities  in 
building  16.  A  number  of  other  organizations  on  Wright- 
Patterson  AFB  also  contract  with  these  companies.  (17) 

Such  programs  can  be  found  on  more  and  more  bases  as 
managers  work  to  meet  training  requirements  (11). 
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Training  Outside  the  Air  Force .  The  private  sector  has 
experienced  similar  trends.  Reaction  is  slow,  but  managers 
are  beginning  to  recognize  the  problems.  The  Forbes  500 
survey  mentioned  earlier  covered  a  five  year  span  1980 
through  1984.  Of  the  173  companies  surveyed,  only  49 
percent  had  implemented  training  programs  before  1984.  In 
1984,  however,  the  number  of  companies  offering  training  to 
end  users  increased  to  72  percent.  (12:183) . 

Experts  continue  to  emphasize  that  training  programs 
are  not  )«eepirg  pace  with  acquisition  programs,  in  or  out  of 
the  Air  Force  (23:324;  25:124;  2:24;  19;  28).  Inadequate 
training  continues  to  be  a  primary  cause  of  less  than 
potential  increases  in  productivity  expected  from  micros 
(23:312,324;  19;  28).  In  her  study  of  microcomputer  usage 
patterns,  Denis  M.  S.  Lee  of  the  Worcester  Polytechnic 
Institute  in  Massachusetts,  cited  a  study  which  estimated, 
"that  between  20%  to  36%  of  microcomputers  end  up  abandoned 
by  users."  Again,  management  attention  was  cited  as  the 
first  step  towards  solution  of  such  problems.  (23:313,324) 
At  least  one  area  requiring  management  attention  is  the 
proper  selection  of  training  methods.  The  next  section 
presents  a  discussion  of  such  alternatives  for  Air  Force 
managers . 
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Training  Methods  -  A  Matrix 

Although  a  number  of  methods  have  been  developed  to 
train  microcomputer  users,  a  few  have  emerged  as  tne  most 
popular  for  training-  and  cost-effectiveness.  These  methods 
can  be  classified  by  a  matrix  of  orientations,  modes,  and 
sources  (20;  8;  21) . 

Orientations  of  Training 

The  three  orientations  refer  to  the  content  of  the 
material  taught:  Computer  literacy  training;  Product- 
oriented  training;  and  Business-oriented  computer  training. 
(20:39-41) 

Computer  Literacy  Training.  Computer  literacy  training 
is  aimed  at  the  basics  of  operating  a  PC  such  as  turning  the 
machine  on  and  off,  loading  the  software,  and  understanding 
basic  operations  of  spreadsheet  and  database  functions. 
(20:39) 

Product-oriented  Training.  Product-oriented  training 
covers  operations  of  individual  software  pac)^ages  such  as 
how  to  find  the  mean  of  a  column  of  values  in  Quattro,  a 
spreadsheet  application.  (20:39-40) 

Business -oriented  Training.  Business-oriented  training 
is  designed  to  teach  the  student  how  to  apply  the  power  of  a 
computer  to  his  or  her  primary  worlc  situations.  One  example 
of  such  training  might  be  how  to  use  spreadsheets  to 
historically  trade  supply  issues  and  forecast  future  demand 
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levels  for  particular  products.  This  particular  orientation 
is  growing  in  popularity  as  it  becomes  more  refined. 
(20:40-41) 

Modes  of  Training 

The  mode  of  training  refers  to  the  medium  through  which 
course  material  is  presented  to  the  student.  There  are  five 
basic  modes  of  PC  training  used  today:  Instructor-led;  Audio 
cassette;  Video;  Computer-based;  and  Interactive  video. 
(8:17-21) 

Instructor- led  Training.  Instructor-led  training,  also 
)cnown  as  classroom  training,  is  the  most  traditional  of  the 
five  modes  of  training.  A  major  advantage  of  this  mode  of 
training  is  the  availability  of  immediate  feedbac)<  for 
students.  Another  advantage  is  the  ease  of  tailoring  the 
course  to  meet  student  needs.  (8:18) 

One  of  the  disadvantages  of  this  mode  is  the  slower 
pace  required  to  meet  varying  skill  levels.  Another 
drawback  is  the  additional  time  required  since  courses 
frequently  cannot  be  broken  into  manageable  segments  as  with 
other  methods;  most  courses  are  based  on  two  to  three  day 
schedules.  If  held  in  the  workplace,  instructor-led 
training  can  be  convenient  and  avoids  travel  costs  for 
students.  But  on  site  training  also  means  that  PCs  won't  be 
available  for  job-specific  output.  (8:18) 
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The  costs  for  instructor-led  training  are  typically 
higher  than  for  other  modes.  For  example,  an  off-site 
course  offered  by  the  American  Management  Association  costs 
more  than  $800  per  person  for  non-members.  Although  this 
mode  is  generally  more  expensive  than  the  other  four,  it  is 
offered  by  more  vendors  than  other  modes;  an  indication  of 
its  persistent  popularity.  (8:18) 

Audio  Cassette  Training.  Audio  cassette  training  is 
among  the  lowest  cost  modes  available;  as  much  as  one  tenth 
the  cost  of  video  training.  Essentially,  this  mode  requires 
the  user  to  listen  to  taped  instructions  while  seated  at  the 
computer.  A  manual  is  usually  provided  for  further 
reference.  (8:19) 

A  major  advantage  of  this  mode  is  that  it  is  self-paced 
and  can  be  broken  into  manageable  segments  to  fit  a  variety 
of  schedules.  Although  answers  to  specific  questions  or 
other  forms  of  feedback  from  an  instructor  aren't  available 
with  this  mode,  if  needed,  the  student  can  review  any 
particular  topic  as  often  as  needed.  Another  advantage  is 
the  lack  of  a  requirement  for  specialized  equipment  -  only  a 
PC  and  a  cassette  player  are  needed.  (8:19) 

Costs  in  1987  for  audio  cassette  training  ranged  from 
approximately  $125  to  $700  per  course.  One  course,  however, 
can  be  used  to  train  many  people.  (8:19) 

Video  Training.  Video  training  offers  a  wide  selection 
of  sources  and  subject  matter.  One  company.  Advanced 
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Systems,  Inc.  (ASI)  based  in  Chicago,  offers  over  3500  video 
training  modules  on  almost  every  PC-related  topic  from  tapes 
of  instructor-led  classes  to  tapes  on  the  latest 
developments  in  computers.  (8:19-20) 

Among  its  advantages,  video  training  offers 
convenience,  speed,  and  a  source  for  future  reference.  Some 
video  courses  include  manuals,  demonstration  disks,  and 
templates  for  users'  keyboards  to  help  them  locate 
frequently  used  keys.  It  is  similar  to  audio  cassette 
training  in  that  the  student  can  train  when  s/he  wants  to 
and  can  review  a  topic  whenever  necessary,  but  instructor 
feedback  isn't  available.  (8:20) 

Of  course,  this  mode  requires  an  investment  in  video 
tape  equipment.  The  costs  for  video  training  can  vary  from 
$500  to  $700  per  package;  but  again,  one  package  can  be  used 
for  a  number  of  students.  (8:20) 

Computer-based  Training.  Computer-based  training,  also 
known  as  CBT,  is  beginning  to  climb  in  popularity  for  its 
cost-effectiveness  and  user-friendliness  (8:20).  CBT 
involves  the  use  of  a  software  package  which  instructs  the 
student  via  a  variety  of  textual  and  graphic  displays  and 
then  tests  him/her  on  the  material  covered  -  all  on  the 
screen  of  the  user's  PC  (32).  More  sophisticated  CBT 
packages  can  analyze  the  student's  responses  and  tailor  a 
combination  of  feedback  and  lesson  review  to  enhance  the 
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student's  understanding  (32).  As  in  audio  and  video 
training  modes,  human  feedback  is  not  available  (32). 

The  costs  for  CBT  packages  vary  widely;  but  experts  and 
managers  agree  that  it  has  proven  to  be  a  cost-effective 
means  of  training  a  large  number  of  users  in  a  relatively 
short  time  (8;  32) .  Experts  also  agree,  however,  that  an 
effective  evaluation  of  whether  CBT  meets  the  needs  of  the 
organization  effectively  in  relation  to  the  costs  involved 
should  be  carefully  undertaken  -  CBT  is  not  for  everyone 
(26:32-35) . 

Interactive  Video  Instruction.  Interactive  video 
instruction  (IVI)  is  the  most  recently  developed  mode  of 
training  for  PC  users.  This  mode  uses  the  computer  and  a 
laser  videodisk  to  communicate  with  the  student  through  the 
keyboard  and,  or  touches  the  screen  to  respond.  (8:21) 

The  user  can  ask  and  answer  questions,  determine  what 
to  cover  next,  and  move  on  to  or  review  subjects  covered  in 
the  training  lesson;  IVI  is  self-paced  and  self-directed. 
Research  shows  that  users'  fascination  with  the  ingenuity  of 
this  mode  of  training  results  in  faster  learning,  greater 
retention,  and  greater  competency  than  with  other  modes. 
(8:21) 

A  major  disadvantage  of  this  mode  is  the  initial 
investment  in  required  hardware:  $10,000  for  a  workstation, 
video-disk  player,  and  touchscreen  monitor.  In  the  future, 
video  course  material  may  be  stored  digitally  on  a  central 
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computer  for  use  at  a  number  of  remote  terminals  which 
should  reduce  costs.  ASI,  the  company  mentioned  earlier, 
offers  this  mode  of  training  by  installing  the  hardware  for 
its  customers  and  leasing  the  software;  the  cost  in  1987  was 
approximately  $100  per  month.  (8:21) 

Sources  of  Training 

There  are  three  sources  of  microcomputer  training 
available  to  managers  today:  Hardware/Software  Vendors; 
Consultants;  and  In-House  specialists  (21:50).  Some  experts 
group  hardware/software  vendor  and  consultant  training  into 
one  category  -  vendor-supplied  training  (20:41).  Although 
discussed  separately  in  detail  in  this  section,  for  purposes 
of  clarification,  the  use  of  Vendor-provided  training 
elsewhere  in  this  thesis  refers  to  both  hardware/software 
vendors  and  consultants.  The  term,  "vendor-provided"  is 
used  by  ATC,  AFCC ,  and  AFSCOAS  to  refer  to  training  provided 
by  either  the  initial  hardware/software  vendor  or  a  third 
party  consultant  including,  in  some  cases,  local  colleges  or 
universities . 

Hardware/ Software  Vendors .  This  source  is  usually  the 
least  costly  of  the  three;  frequently  free  with  the  purchase 
of  equipment  and,  or  software  (21:50).  Training  received  is 
generally  on  an  initial  familiarization  basis  and  includes 
little  or  no  follow-up  without  additional  charges  (11) . 
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Initial  training  most  often  includes  a  brief 
orientation  by  a  technician  supplemented  with  self-paced 
packages  of  manuals,  tutorial  disks,  and,  or  video  tapes. 
Such  training  can  be  offered  at  the  vendor's  place  of 
business  or  on  site.  (21:50) 

A  major  drawback  to  this  source  of  training  is  the  lack 
of  feedback  from  an  instructor  or  knowledgeable  person  once 
the  training  has  been  completed.  Also,  users  can  spend  a 
great  deal  of  time  looking  for  the  answers  in  manuals  or  on 
disks  or  tapes,  thus  wasting  valuable  work  time.  And  such 
answers  may  not  always  by  the  best  solution  to  the  problem. 
(21:50) 

Customp'  raining  is  being  recognized  by  vendors, 
however,  a  growing  in  importance  to  their  ability  to  sell 
their  product  (30:85-6) .  Fifty-eight  percent  of  the  vendors 
sr.rveyed  by  Work  System  Associates,  Inc.,  in  Framingham,  MA, 
indicated  that  customer  training  was,  "...either  essential 
(could  not  sell  the  product  without  it)  or  very 
important..."  (30:86) 

Although  the  costs  are  usually  much  lower  for  this 
source  of  training,  its  limitations  are  often  the  reasons 
organizations  turn  to  other  sources  of  training.  (21:50-51) 
Consultants .  Third  party  consulting  has  evolved  into  a 
fast-growing  industry  (30:84).  The  major  advantage  to  this 
source  of  microcomputer  training,  which  typically  involves 
off-site  classroom  training,  is  the  lack  of  necessity  to  use 
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in-house  resources.  This  advantage  should  be  considered 
carefully  and  compared  to  the  costs  of  such  training  which 
can  be  significantly  higher  than  other  sources.  (10:50; 
8:18).  Also,  unless  recorded  in  some  useful  manner,  the 
material  covered  cannot  be  accessed  for  future  reference 
(21:50). 

Some  consulting  companies  are  responding  to  this  lacl< 
of  reference  by  providing  materials  which  students  can  take 
back  to  the  workplace  (7;  17).  The  companies  providing 
training  to  ASD,  for  example,  provide  manuals  which  are 
written  so  users  can  find  answers  quicker  and  understand 
them  easier  than  with  the  original  manuals  provided  by 
hardware  or  software  companies  (17;  7). 

In-House  Training.  Larger  organizations  often  find 
that  the  development  of  an  in-house  training  function  is 
more  cost-effective  and  easier  to  tailor  training  to 
specific  organizational  needs.  In-house  training  is  very 
often  less  expensive  than  training  provided  by  consultants. 
It  can  also  offer  more  flexibility  and  reliability  in 
scheduling  than  outside  training  sources.  (21:50;  20:41) 

The  potential  disadvantages  of  this  source  of  training 
are  the  requirements  placed  on  the  organization's  resources 
-  time,  personnel,  and  money.  Separate  facilities, 
additional  personnel,  and  separate  equipment  are  required 
for  in-house  training.  (21:50;  20:41) 
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These  disadvantages  could  be  considered  important  only 
in  the  short  run.  If  the  training  of  employees  is 
considered  an  investment  in  increased  productivity,  and  the 
return  on  the  investment  is  carefully  calculated,  the 
disadvantages  can  be  outweighed.  This  assumes,  of  course, 
that  the  in-house  training  function  is  managed  effectively. 
(21:50;  20:41) 

All  of  these  sources  of  training  will  soon  be  able  to 
offer  any  of  the  five  modes  geared  to  any  of  the  three 
orientations  discussed  above.  Customer  demands  will  require 
and  technology  will  make  possible  an  increased  number  of 
alternatives  for  managers  to  consider  in  meeting  their 
employee's  training  needs.  (30;  21;  20;  32) 

Summary 

The  rapid  growth  in  microcomputer  hardware  in  the  Air 
Force  presents  significant  problems  to  Air  Force  management. 
This  chapter  discussed  the  importance  of  proper  planning  for 
the  growth  of  microcomputers  and  adequately  addressing 
subsequent  training  requirements.  A  main  point  should  be 
stressed  here  -  more  and  better  formal  training  programs  are 
needed  and  should  be  focused  more  towards  the  end  users  of 
microcomputers . 

With  this  focus  in  mind,  the  results  of  efforts  by  both 
the  Air  Force  and  private  sector  organizations  to  meet  PC 
training  needs  were  discussed.  Managers  of  all  modern 
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organizations  which  utilize  microcomputers  acknowledge  the 
need  to  further  improve  the  training  offered  to  their  end- 
user  employees. 

Also  discussed  were  the  orientations,  modes,  and 
sources  of  training  currently  available  to  modern 
organizations  were  presented  and  analyzed.  These  elements 
form  a  training  matrix  from  which  the  manager  is  able  to 
choose  the  content,  the  method,  and  the  source  of 
microcomputer  training.  Such  a  matrix  permits  the  manager 
to  tailor  training  to  the  users'  as  well  as  the 
organization's  needs  (8;  20). 

Business-oriented  training,  which  teaches  employees  to 
use  microcomputers  to  solve  real  world  problems,  is  growing 
as  users  move  upward  from  basic  skills  training  to  software 
mastery  training.  While  instructor-led  classroom  training 
maintains  a  foothold  in  the  computer  training  industry,  more 
innovative  modes  (or  methods)  of  training,  such  as  computer- 
based  training  (CBT)  and  interactive  video  instruction  (IVI) 
are  gaining  in  popularity  as  they  become  more  effective  in 
terms  of  training  results  and  costs. 

Since  consultant  training  usually  means  high  costs, 
companies  which  sell  hardware  and  software  are  offering 
improved  training  as  a  discriminator  when  competing  for 
organizations'  business.  But  larger  organizations, 
including  the  Air  Force,  are  finding  that  establishing  in- 
house  training  can  mean  lower  costs  for  training  which  can 
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be  tailored  to  fit  their  needs.  Some  bases  are  taking  more 
innovative  approaches  by  exploring  the  use  of  local  colleges 
or  u,' .iversities  as  sources  of  PC  training;  the  potential 
benefits  of  such  programs  are  unique. 

The  next  chapter  will  present  the  methodology  used  in 
this  thesis  to  evaluate  two  sources  and  methods  of 
government  microcomputer  training.  Again,  the  focus  of  the 
thesis  is  on  two  primary  sources  -  in-house  and  vendor- 
provided,  and  one  primary  method  -  classroom.  As  mentioned 
earlier,  vendor-provided  training  refers  to  training 
provided  to  government  employees  through  a  contract.  A 
vendor  may  be  a  hardware  or  software  vendor  or  a  third  party 
consultant.  In  this  thesis,  the  vendor  is  a  third  party 
consultant.  Chapter  III  will  also  present  how  data  will  be 
evaluated  for  a  third  source  of  PC  training  -  colleges  and 
universities,  as  well  as  the  three  methods  of  PC  training  - 
classroom,  CBT,  and  video-based. 


30 


III.  Methodology 


Introduction 

This  chapter  will  discuss  the  methodology  used  to 
evaluate  sources  and  methods  of  microcomputer  training  for 
government  employees.  The  objectives  of  this  thesis  are 
presented  first.  The  steps  taken  to  perform  the  cost 
benefit  analysis  are  discussed  next.  This  is  followed  by  a 
discussion  of  the  content  and  structure  of  the  survey  used 
to  collect  benefit  data.  The  last  sections  of  this  chapter 
cover  how  survey  responses  were  treated  and  the  responses 
which  were  expected  from  the  survey  for  each  benefit. 

Objectives 

The  primary  objective  of  this  thesis  was  to  compare 
the  two  primary  sources  of  government  microcomputer 
training,  in-house  and  commercial.  The  comparison  focused 
on  the  most  prevalent  method  of  PC  training  used  in  the  Air 
Force,  classroom  instruction.  A  cost  analysis  identified 
costs  associated  with  each  source.  Benefits  and  costs  were 
used  to  compute  benef it-per-dollar-spent  ratios  for  each 
source  of  training. 

A  secondary  objective  of  the  thesis  was  to  rank  each 
particular  method/source  combination  according  to  the  level 
of  benefits  achieved.  As  mentioned  in  chapter  II,  any 
orientation  or  mode  (method)  of  training  may  be  available 
from  any  particular  source  of  PC  training.  Data  on  private 
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sources  as  well  as  the  other  two  methods  of  PC  training  was 
also  collected  and  analyzed.  A  survey-questionnaire  of 
government  employees  was  used  to  collect  the  data  which 
represent  the  benefits  of  each  of  three  methods:  classroom, 
video,  and  CBT.  The  responses  from  the  survey  were  averaged 
according  to  source  and  method  and  used  to  - ank  each 
combination. 

Cost  Benefit  Analysis 

The  primary  instrument  used  to  evaluate  in-house  and 
commercial  sources  of  classroom  training  was  a  cost  benefit 
analysis.  Direct  and  applicable  indirect  costs  for  each 
source  of  training  were  computed  to  derive  a  cost-per- 
training  hour.  The  benefits  used  in  the  analysis  were 
measured  by  a  survey  of  government  employees  on  Wright- 
Patterson  AFB,  Ohio.  Benefit-to-cost  ratios  were  used  to 
used  evaluate  the  two  sources. 

Cost  Analysis .  All  costs  which  are  identical  to  both 
sources  were  excluded  from  the  cost  analysis.  For  example, 
time  away  from  primary  duties  is  not  reflected  in  the  costs 
of  either  source. 

Costs  for  the  in-house  source  were  provided  by  HQ  AFLC 
personnel  responsible  for  government  accounting  system 
records.  Examples  of  these  costs  include  supplies, 
utilities,  personnel,  travel,  and  facilities  maintenance 
costs . 
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Direct  costs  for  the  training  contracts  were  provided 
by  ASD  personnel  responsible  for  managing  the  contracts. 
Indirect  costs  were  also  provided  by  ASD  personnel  who 
maintain  government  accounting  system  records. 

Benefit  Analysis .  Levels  of  benefits  for  both  sources 
and  all  methods  offered  were  estimated  by  using  data 
gathered  in  a  survey  of  government  employees  working  on 
Wright-Patterson  AFB,  Ohio.  The  next  section  discusses  how 
responses  were  converted  to  represent  levels  of  three 
benefits:  overall  value  to  the  employee,  training 
effectiveness,  and  level  of  competence  achieved  as  results 
of  various  methods  and  sources  of  microcomputer  training. 

Survey 

Appendix  A  is  a  copy  of  the  survey  given  to  540 
government  employees  of  Wright-Patterson  AFB,  Ohio.  One- 
half  of  the  540  surveys  were  sent  to  employees  who  had  taken 
at  least  one  microcomputer  training  course  at  the  Air  Force 
Logistics  Command  in-house  training  facility  in  building  110 
of  area  C.  The  other  270  surveys  were  sent  to  employees  who 
had  taken  at  least  one  course  offered  by  vendors  in  the 
Dayton  area,  the  primary  source  for  microcomputer  training 
for  the  Aeronautical  Systems  Division  (ASD)  of  Air  Force 
Systems  Command  (17) .  The  survey  was  approved  by  the  Air 
Force  Military  Personnel  Center  which  assigned  USAF  SCN  89- 
59  (survey  control  number) . 
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Sample  Size.  The  author  chose  the  sample  size  of  540 
to  measure  a  population  of  approximately  19000  employees. 
Based  on  discussions  with  officials  at  the  base  civilian 
personnel  office,  the  historically  expected  response  rate 
was  25%,  i.e.,  135  surveys  were  expected  to  be  returned  (7). 
A  sample  of  135  from  the  population  of  19000  would  produce 
results  with  a  confidence  interval  of  90%  and  an  error  rate 
of  +10%. 

Survey  Structure.  The  survey  is  composed  of  three 
questions  which  ask  the  respondent  to  assign  a  value  between 
0  and  100,  in  increments  of  10,  to  each  of  nine  elements  per 
question.  The  respondent  is  asked  to  assume  that  100 
represents  the  ideal  level  of  the  particular  benefit  being 
addressed.  This  structure  is  designed  to  force  the 
respondent  to  compare  all  methods  of  training  simultaneously 
for  the  same  benefit.  The  following  structure  was  used  for 
all  three  questions: 


IN-HOUSE  (officially  provided  by  government  employees) : 

_  a.  Classroom  training  (traditional  student- 

instructor) 

_  b.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  c.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

COMMERCIAL  (through  use  of  government  contract) : 

_  d.  Classroom  training  (traditional  student- 

instructor) 

_  e.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 
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f.  Computer-based  training  (computer  software 
"instructs”  and  prompts  student  for  input) 


PRIVATE  (through  your  own  efforts  or  off-duty): 

_  g.  Classroom  training  (high  school,  college, 

university) 

_  h.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  i.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

The  use  of  the  word  commercial  in  the  survey  refers  to 
training  provided  through  a  government  contract.  The  third 
source,  private,  was  added  to  measure  how  employees  rate 
off-duty  training.  Such  information  could  be  used  to 
study  the  possibility  of  expanding  support  of  off-duty 
microcomputer-related  training.  Chapter  II.  mentioned  that 
this  particular  avenue  is  being  explored  by  other  bases 
which  are  located  near  under-graduate  or  graduate 
educational  institutions. 

Expected  Responses .  The  following  sections  include  the 
expected  responses  for  each  method  and  source  of  training 
addressed  by  the  survey.  To  develop  data  for  the  benefit 
measured,  responses  for  each  item  were  averaged. 

The  means  of  each  response  were  tested  for  statistical 
significance.  The  Standard  Normal  Random  Variable,  or  Z- 
test  were  performed  under  the  assumption  that  the  data  were 
collected  from  normally  distributed  populations.  A 
statistical  significance  level  of  five  percent  was  used  for 
the  two-tailed  test  to  compare  means  ratings  for  each 
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method/source  combination.  The  Mann-Whitney  test  is  a  non- 
parametric  test  which  does  not  require  the  assumption  of 
normally  distributed  data.  The  Mann-Whitney  test  was 
performed  only  to  verify  differences  (also  at  the  five 
percent  level)  in  means  for  in-house,  vendor-provided,  and 
private  source  classroom  training.  These  tests  were 
perfprmed  to  determine  if  the  average  responses  for  each 
method/source  combination  were  significantly  different, 
statistically  speaking.  Test  formulas  and  guidelines  can  be 
found  in  the  reference  cited  in  the  bibliography  (27) .  The 
Mann-Whitney  ranked  sums  were  produced  using  a  generic 
statistical  computer  program. 

Value  of  Training  to  the  Employee .  Question  1.  asks 
the  employee  to  identify  and  evaluate  each  type  of  training 
the  employee  has  taken.  This  question  essentially  measures 
employee  satisfaction  as  a  result  of  the  method  of  training 
used.  Question  1.: 


1.  This  question  is  designed  to  do  two  things:  a) 
determine  which  types  of  training  you  have  completed; 
and  b)  how  you  would  rate  the  types  of  training  you 
have  received  in  terms  of  overall  value  to  you  as  a 
government  employee.  By  overall  value,  we  mean  the 
degree  to  which  the  training  made  you  more  productive, 
versatile,  and,  or  effective  in  your  job.  Remember, 
100  represents  the  ideal  level  of  value  to  you  as  an 
employee.  Assign  a  number  only  to  those  types  of 
training  you  have  completed  (even  if  you've  only  had 
one)  . 
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The  expected  number  of  responses  for  classroom  training 
from  all  three  sources,  in-house,  commercial,  and  private, 
was  expected  to  exceed  60%  for  all  responses.  Based  on  the 
popularity  of  classroom  training,  the  percentages  for  all 
three  sources  of  classroom  training  were  also  expected  to  be 
high,  i.e.,  above  70  percent.  The  order  of  value  by  source 
was  expected  to  be  in-house,  commercial,  and  private. 

The  least  popular  method  was  expected  to  be  video-based 
training.  The  expected  order  of  value  by  source  was 
commercial,  private,  and  in-house.  Responses  for  these 
items  were  expected  to  be  below  30%.  The  number  of 
employees  who  have  taken  such  training  was  expected  to 
comprise  less  than  25%  of  the  sample. 

The  remaining  items  associated  with  CBT  were  expected  to 
receive  responses  of  between  30%  and  60%.  The  number  of 
employees  who  have  taken  CBT  training  was  expected  to  be  only  ^ 
slightly  higher  than  the  number  who  have  taken  video-based 
training.  The  order  of  value  by  source  was  expected  to  be 
commercial,  in-house,  private. 

Training  Effectiveness .  Question  2.  asks  the  employee 
to  determine  the  amount  of  material  the  employee  effectively 
learned  per  hour  of  training  as  a  result  of  the  method  and 
source  of  training  rather  than  the  material  content  of  the 
course.  Question  2.: 
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2.  This  question  asks  you  to  determine  how  much  you 
can  effectively  learn  from  a  course  as  a  result  of  the 
type  of  training  used.  Assume  that  100  represents  the 
most  material  you  could  effectively  learn  in  one  hour 
of  training.  For  each  of  the  types  of  training  you 
have  taken,  assign  a  number  which  represents  the  amount 
of  material  you  feel  can  be  learned  for  each  hour  of 
training. 


The  highest  responses  were  expected  to  again  be  for 
classroom  training  from,  in  order  of  effectiveness,  in- 
house,  commercial,  and  private.  Responses  for  this  method 
were  expected  to  be  between  70%  and  100%. 

Video-based  training  responses  were  expected  to  be 
between  50%  and  90%.  Although  no  specific  order  was 
expected,  video-based  training  from  a  commercial  source  was 
expected  to  receive  the  greatest  number  of  responses  among 
the  three  sources. 

Responses  between  10%  and  60%  were  expected  for 
computer-based  training.  In-house  was  expected  to  receive 
the  most  responses  and  private,  the  least  among  sources  for 
this  method  of  training. 

Employee  Competence.  Question  3.  asks  the  employee  to 
determine  his/her  level  of  competence  as  a  result  of  the 
method  and  source  of  training  taken.  Question  3.: 


3.  This  question  asks  you  to  determine  your  level  of 
competence  upon  completion  of  each  type  of  training  you 
received.  For  example,  if  you  took  a  course,  but  felt 
you  could  only  master  60%  of  the  course  material 
because  of  the  type  of  training  used,  assign  a  60. 
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Again,  classroom  training  was  expected  to  receive  the 
highest  responses  ranging  from  70%  to  100%.  The  expected 
order  by  source  was  in-house,  commercial,  and  private. 

Computer-based  training  was  expected  to  receive  the 
second  highest  responses  ranging  from  60%  to  80%.  The 
expected  order  was  the  same  as  for  classroom  training:  in- 
house,  commercial,  and  private. 

The  average  expected  response  for  the  level  of 
competence  achieved  through  video-based  training  was  50%. 
Although,  no  particular  order  was  expected,  private  and 
commercial  video-based  training  were  expected  to  be  rated 
higher  than  in-house  video-based  training. 

The  next  chapter  presents  the  findings  of  this 
particular  research  effort. 


Introduction 


IV.  Findings 


The  chapter  will  present  the  findings  of  the  cost  and 
benefit  analyses  performed  to  evaluate  which  sources  of  PC 
training  are  most  cost-effective  for  the  Air  Force.  Survey 
response  is  discussed  first,  followed  by  an  analysis  of 
employee  training  experience.  An  evaluation  of  the  ratings 
data  for  benefits  of  each  source  and  method  of  training  is 
discussed  next. 

Each  benefit  is  discussed  in  terms  of  the  averages 
of  all  ratings  for  each  particular  method,  and  the  number  of 
employees  selecting  one  method  over  another.  Appendix  B 
contains  spreadsheet  printouts  of  the  ratings. 

The  first  benefit  discussed  is  overall  value  to  the 
government  employee  of  training  received  from  in-house 
sources,  commercial  sources  (vendor-provided),  and  private 
sources.  Next,  employee  ratings  for  effectiveness  of 
training  from  each  source  are  discussed.  This  is  followed 
by  a  discussion  of  ratings  for  the  levels  of  competency 
achieved  from  training  from  each  of  the  method/source 
combinations.  Tests  for  statistical  significance  of  the 
data  follow  the  discussions  for  each  benefit.  Appendices  C, 
D,  and  E  contain  statistical  test  results  for  overall  value,’ 
effectiveness,  and  competency  achieved,  respectively. 
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The  next  section  includes  results  of  a  cost  analysis  of 
each  of  the  two  primary  sources  of  PC  training  used  by  the 
Air  Force  today  are  presented.  Costs  for  in-house  training 
are  outlined  first,  followed  by  costs  for  commercial,  or 
vendor-provided  training. 

Finally,  the  last  section  in  this  chapter  discusses  the 
ranking  of  each  method  and  each  source  of  PC  training.  In- 
house  and  commercial  sources  of  classroom  training  are 
compared  first,  then  all  sources  and  methods  are  ranked. 

Survey  Responses 

The  response  rate  predicted  by  officials  at  the  2750th 
Air  Base  Wing  Civilian  Personnel  Office  (of  25  percent)  was 
not  significantly  different  from  the  actual  response  rate  of 
34  percent  (31) .  Of  the  540  personnel  surveyed,  184 
responded.  Of  the  184  received,  seven  responses  indicated 
that  the  respondents  could  not  understand  the  survey  well 
enough  to  complete  it.  Of  the  remaining  177  surveys,  26 
contained  responses  for  questions  two  and  three  concerning 
training  which  the  respondents  had  not  indicated  they  had 
taken.  These  26  surveys  were  not  used  to  compile  the 
statistical  data  for  the  findings  in  this  chapter,  thus  all 
findings  are  based  on  responses  received  from  151  government 
employees.  The  results  of  the  sample  of  151  from  the 
estimated  population  of  19000  should  be  accurate  to  within 
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+7  percent  of  the  population  parameters  with  a  95  percent 
level  of  confidence. 

Survey  data  was  collected  and  input  into  microcomputer 
database  and  spreadsheet  programs  for  data  aggregation  and 
analysis.  Appendix  B  contains  electronic  spreadsheet 
printouts  of  the  151  responses  for  questions  1,  2,  and  3., 
as  well  as  some  limited  statistical  data  for  each  question 
item  which  includes  the  number  of  responses,  the  maximum  and 
minimum  values,  and  the  average  of  and  standard  deviation 
for  each  question  item.  Other  data  mentioned  elsewhere  in 
this  chapter  came  from  various  interactive  inquiries  made 
with  the  database  program. 

Employee  Training  Experience 

The  first  question  of  the  survey  served  a  dual  purpose: 
to  identify  which  training  methods  and  sources  the  151 
employees  had  been  exposed  to  and  what  their  reactions  were. 

Nearly  all  of  the  respondents,  150,  had  experienced  at 
least  one  classroom  PC  training  coiirse.  One  hundred  twenty- 
eight  (128)  employees  had  taken  at  least  one  in-house 
classroom  training  course.  Almost  as  many,  108,  had  taken 
classroom  training  from  a  commercial  source.  The  next 
largest  group  of  employees  (86)  had  taken  classroom  PC 
training  from  a  private  source,  e.g.,  a  university  or 
college.  Forty-two  (42)  employees  had  classroom  training 
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only.  Table  I.  shows  the  number  of  employees  who  had  taken 
each  method  of  PC  training  by  source. 


Table  I.  Employee  Training  Experience  by  Method 


Source 

Method 

In-House 

Classroom 

In-House 

Video 

In-House 

CBT 

Contract 

Classroom 

Contract 

Video 

Co  itract 

CBT 

Private 

Classroom 

Private 

Video 

Private 

CBT 

Number  of  Employees 

128 

50 

84 

108 

39 

62 

86 

42 

71 


Those  respondents  who  had  taken  video-based  training 
comprised  the  smaller  groups.  Fifty-seven  (57)  respondents 
indicated  they  had  taken  at  least  one  video-based  PC 
training  course.  Employees  who  had  taken  vendor-provided 
video  training  were  in  the  smallest  group  of  39. 

A  significant  number  of  employees  had  been  exposed  to 
CBT  training.  One  hundred  three  (103)  employees  had  taken 
at  least  one  computer  course  by  CBT.  More  employees  had 
taken  CBT  training  from  in-house  sources  than  from 
contractors  or  universities;  this  was  the  third  largest 
group  categorized  by  method. 

Thirty-three  (33)  employees  indicated  that  they  had 
taken  all  three  methods  of  training  from  all  three  sources. 
Of  the  151  respondents,  136  had  taken  at  least  one  course 
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from  an  in-house  source,  112  from  a  government  contracted 
source,  and  103  from  a  private  source. 

Most  of  the  respondents  had  been  exposed  to  a  variety 
of  training  methods  and  sources.  Eighteen  (18)  had  taken 
training  only  from  an  in-house  source  and  10  had  training 
only  from  a  commercial  source.  None  of  the  respondents  had 
training  only  from  a  private  source.  Fifteen  (15)  employees 
indicated  they  had  taken  no  training  from  an  in-house 
source,  39  had  no  training  from  a  commercial  source,  and  48 
had  no  training  from  a  private  source. 

Overall  Value  to  the  Employee 

This  particular  benefit  was  used  to  measure  the  value 
to  each  employee's  job  of  each  method  and  source  combination 
of  microcomputer  training.  Value  refers  to  the  degree  to 
which  the  training  made  the  employee  more  productive, 
versatile,  and,  or  effective  in  his/her  job.  Table  II. 
summarizes  the  average  ratings  for  each  of  the  methods  and 
sources  surveyed.  Please  note  that  the  numbers  represent 
the  percentage  of  the  ideal  level  of  value. 

Primary  Conclusions .  Two  primary  conclusions  are 
apparent  from  observing  Table  II.  First,  regardless  of 
source,  the  preference  of  delivery  method  is  classroom,  CBT, 
then  video,  in  terms  of  perceived  value  to  the  employee. 
Second,  regardless  of  the  method,  there  is  no  discernible 
difference  in  the  perception  of  value  with  respect  to  in- 
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house,  contract,  or  private  sources  of  PC  training.  The 
reader  should  also  note  that  although  average  ratings  for 
the  three  sources  of  classroom  training  were  in  the  70s, 
actual  modal  responses  were  in  the  80  to  90  range  for  in- 
house  and  commercial  sources  and  in  the  90  to  100  range  for 
private  sources.  This  should  lead  to  the  conclusion  that 
the  level  of  value  to  employees  is  higher  than  indicated  by 
the  mean  responses  for  each  method  and  source  of  PC 
training.  A  similar  point  is  made  regarding  means  and  modes 
for  other  methods  of  training.  Modal  responses  can  be  found 
in  the  Frequency  Distribution  tables  in  Appendix  B. 


Table  II. 

Average  Ratings 

for  Overall 

Source 

Method 

Average  Rat 

In-House 

Classroom 

74.375 

In-House 

Video 

46.700 

In-House 

CBT 

69.643 

Contract 

Classroom 

75.324 

Contract 

Video 

50.946 

Contract 

CBT 

62.742 

Private 

Classroom 

76.419 

Private 

Video 

51.429 

Private 

CBT 

63.028 

In-House  Training.  The  average  response  for  overall 
value  of  in-house  classroom  training  was  the  third  highest 
among  all  methods.  The  modal  response  of  80  percent  was  the 
same  for  in-house  classroom  as  for  commercial  classroom 
training.  In-house  training  as  a  source  of  PC  training, 
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however,  received  a  lower  rating  for  all  three  methods. 

Forty  (40)  percent  of  those  employees  who  had  taken  training 
from  a  government  source  rated  it  above  80  as  compared  to  64 
percent  of  the  employees  who  had  rated  commercial  training 
80  or  higher. 

Of  the  33  respondents  who  had  taken  all  three  methods 
of  training  from  all  three  sources,  five  indicated  that  in- 
house  classroom  training  was  more  valuable  to  their  jobs 
than  vendor-provided  classroom  training.  Twice  as  many, 

(10)  indicated  vice-versa.  Nearly  half  (14),  however,  felt 
in-house  classroom  training  was  more  valuable  than  private 
classroom  training.  Sixteen  (16)  of  the  33  rated  in-house 
and  commercial  equally. 

Eighty-six  (86)  employees  had  taken  classroom  training 
from  both  an  in-house  and  a  commercial  source.  Thirty-one 
(31)  of  those  86  responses  rated  commercial  classroom 
training  over  in-house  for  overall  value,  15  rated  in-house 
higher,  and  40  rated  both  equally. 

In-House  video  training  received  the  lowest  average 
rating  for  overall  value  among  all  methods  and  sources.  The 
modal  response  for  this  method  was  50  percent.  In  fact, 
thirty-two  (32)  of  the  50  people  who  had  taken  video 
training  from  an  in-house  source  rated  it  at  or  below  50 
percent . 

More  personnel  (84)  had  taken  CBT  training  from  an  -n- 
house  source  than  from  the  other  two.  The  modal  response 
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for  this  method  was  70  percent;  42  of  the  84  employees  who 
had  taken  this  method  of  PC  training  rated  it  higher  than  70 
percent  for  overall  value. 

Commercial  Training.  The  average  ratings  for 
commercial  training  were  higher  than  in-house  training  but 
lower  than  private  training  for  all  three  methods  in  terms 
of  overall  value  to  the  employees*  jobs.  Of  the  69 
employees  who  had  taken  classroom  training  from  all  three 
sources,  the  largest  subgroup  of  26  rated  commercial 
training  higher  than  in-house  and  23  people  rated  commercial 
higher  than  private. 

Employee  preference  for  video  training  did  not  indicate 
any  particularities;  ratings  were  almost  equally  dispersed 
between  all  levels  of  value.  Among  those  employees  who  had 
taken  CBT  training  from  two  or  more  sources,  commercial  was 
rated  higher  than  in-house  and  private  sources.  The 
response  for  commercial  CBT  training  was  bi-modal  at  70  and 
80.  Thirty-eight  (38)  percent  of  those  employees  who  had 
taken  commercial  CBT  training  rated  it  at  or  above  80.  This 
compared  to  44  percent  of  those  who  had  rated  CBT  training 
from  a  private  source  at  or  above  80. 

Private  Training .  The  modal  response  for  private 
classroom  training  was  90,  as  compared  to  80  for  both  in- 
house  and  commercial  classroom  training.  Nearly  64  percent 
of  those  employees  who  had  taken  classroom  training  from  a 
private  source  rated  it  above  80  percent  in  terms  of  overall 
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value  to  their  jobs.  This  compares  to  similar  percentages 
for  in-house  and  commercial  of  60  and  63,  respectively. 
Classroom  training  from  private  sources  received  the  highest 
average  rating  among  all  methods  (76.419  percent). 

Video  training  from  a  private  source  was  rated  higher 
than  commercial  or  in-house  training  as  well,  though  ratings 
were  almost  as  equally  dispersed  as  those  for  commercial 
video  training. 

Employees  also  rated  private  source  CBT  higher  than 
commercial  or  in-house.  The  modal  response  for  this  method 
was  80  percent.  Forty-five  (45)  percent  of  employees  who 
had  taken  this  method  rated  it  higher  than  80  percent  of  the 
ideal  level  of  value  to  the  job. 

Statistical  Tests  for  Overall  Value  Ratings 

The  average  ratings  were  tested  for  statistical 
significance  to  determine  if  there  are  indeed  significant 
differences  among  the  methods.  As  mentioned  earlier  in 
chapter  III.,  the  tests  included  the  Z  test  and  the  Mann- 
Whitney  test.  The  Mann-Whitney  test  was  performed  only  to 
verify  differences  in  means  for  in-house,  vendor-provided, 
and  private  source  classroom  training.  Results  of  each  test 
are  at  Appendix  C. 

Z  tests  indicated  that  there  were  no  statistical 
differences  among  sources  of  training  for  any  of  the 
methods  at  the  five  (5)  percent  level  of  significance. 
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Since  the  differences  between  methods  were  apparent,  no 
tests  were  performed  to  compare  one  method  to  another.  The 
Mann-Whitney  tests  also  indicated  that  mean  ratings  for  in- 
house  and  commercial  classroom  training  were  not 
statistically  different.  Essentially,  this  means  that  in- 
house  classroom  can  be  ranked  equally  with  commercial 
classroom  in  terms  of  overall  value  to  an  employee’s  job. 

Recall  that  a  primary  focus  of  this  thesis  was  to  evaluate 
in-house  versus  vendor-provided  classroom  training.  The 
Mann-Whitney  test  results  indicate  that,  based  on  this 
sample,  no  difference  exists  between  the  two  as  to  overall 
value  to  an  employee's  job. 

Mann-Whitney  tests  comparing  means  for  in-house  and 
private  source  classroom  training  indicated  that  overall 
value  ratings  for  private  source  classroom  training  were 
higher  than  those  for  in-house  below  the  88  percent  level  of 
probability.  Private  source  ratings  for  overall  value  were 
also  higher  than  commercial  ratings  below  the  88  percent  level. 
These  results  suggest  some  difference  may  exist  between 
private  and  the  other  two  sources  of  classroom  training,  but 
more  evidence  is  needed  before  such  a  conclusion  can  be 
made. 

Effectiveness  of  Training 

This  benefit  was  measured  by  the  amount  of  material 
a  student  could  effectively  learn  in  one  hour  of  PC  training 
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as  the  result  of  the  method  and  source  of  the  training,  not 
the  course  material.  Each  respondent  was  asked  to  assign  a 
number  which  represents  the  percentage  of  the  most  that 
could  be  learned  in  one  hour  of  training.  Table  III. 
contains  the  averages  of  ratings  for  training  effectiveness 
of  each  method. 

Primary  Conclusions .  Primary  conclusions  similar  to 
those  for  Table  II  can  be  drawn  from  Table  III.  First, 
regardless  of  source,  the  preference  of  delivery  method  is 
classroom,  CBT,  then  video,  in  terms  of  effectiveness 
perceived  by  employees.  Second,  regardless  of  the  method, 
there  is  no  practical  difference  in  employees'  perception  of 
effectiveness  of  in-house,  contract,  or  private  source;  of 
PC  training.  The  reader  is  cautioned  again  that  mean 
ratings  presented  in  Table  III  do  not  necessarily  indicate 
true  levels  of  effectiveness.  Modal  responses  for  each 
method/source  combination  may  be  different  than  the  average 
ratings  which  indicates  that  actual  effectiveness  of  each 
method/source  may  be  higher  or  lower.  Further  data  about 
modal  responses  can  be  found  in  Frequency  Distribution 
tables  in  Appendix  B. 
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Table 

III.  Average  Ratings 

for  Effectiveness 

Source 

Method 

Average  Rating 

In-House 

Classroom 

69.258 

In-House 

Video 

46.500 

In-House 

CBT 

59.048 

Contract 

Classroom 

73.333 

Contract 

Video 

49.103 

Contract 

CBT 

59.919 

Private 

Classroom 

75.558 

Private 

Video 

55.476 

Private 

CBT 

66.408 

In-Hou3e  Training.  The  average  ratings  for 
effectiveness  of  in-house  methods  of  micro  training  were 
lower  than  commercial  and  private  sources  by  a  larger  margin 
than  ratings  of  overall  value  in  question  1.  The  majority 
(44  of  85)  of  those  employees  who  had  taken  classroom 
training  from  both  in-house  and  commercial  sources  rated  the 
two  equally  for  effectiveness.  Fifteen  (15)  of  the  85  rated 
in-house  higher.  Thirty-six  (36)  of  the  81  people  who  had 
taken  classroom  training  from  in-house  and  private  sources 
assigned  equal  ratings  to  both  methods.  Sixty-eight  (68) 
personnel  had  taken  classroom  training  from  all  three 
sources.  Thirty-two  (32)  gave  the  same  rating  to  all  three 
sources;  five  (5)  rated  in-house  higher  than  the  other  two. 

Effectiveness  of  in-house  video  training  was  rated 
lower  than  video  training  from  the  other  two  sources.  The 
lower  ratings  followed  the  same  pattern  found  in  ratings  for 
overall  value  in  question  1.  The  modal  response  for  this 


method  was  60  percent.  Forty  (40)  of  the  50  employees  who 
had  taken  video  training  from  an  in-house  source  rated  it 
lower  than  60  percent  for  training  effectiveness.  Responses 
for  the  other  two  sources  of  video  training  also  followed  a 
similar  pattern,  however. 

In-house  CBT  training  was  also  rated  lower  than  CBT 
training  from  the  other  two  sources.  The  modal  response  of 
70  percent  was  one  level  lower  than  the  modes  for  commercial 
and  private  CBT  training  of  80  percent.  More  than  one-half 
of  the  employees  who  had  taken  CBT  training  from  two  or  more 
sources  rated  in-house  equally  with  the  other  source (s). 

Commercial  Training.  The  relationship  of  ratings  for 
commercial  sources  of  classroom  PC  training  to  ratings  for 
in-house  and  private  sources  changed  for  question  2.  The 
differences  in  average  ratings  increased  between  in-house 
and  commercial  by  approximately  four  (4)  percent  and 
decreased  between  commercial  and  private  by  approximately 
one  and  one-half  (1.5)  percent.  Twenty-six  (26)  of  the  85 
employees  who  had  taken  classroom  training  from  in-house  and 
commercial  sources  rated  commercial  higher,  44  rated  them 
equal  for  effectiveness.  Forty-one  (41)  employees  who  had 
taken  classroom  training  from  commercial  and  private  sources 
rated  both  equally,  14  rated  commercial  higher. 

Ratings  for  effectiveness  of  commercial  video  training 
also  reflected  a  change  in  the  relationships  to  ratings  for 
in-house  and  private  sources.  Average  effectiveness  ratings  for 
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commercial  and  in-house  were  closer  than  value  ratings  in 
question  1.  The  frequency  of  ratings  for  effectiveness 
levels  of  60  percent  and  below  increased  as  it  did  for  in- 
house  video  training.  Seven  (7)  of  the  39  personnel  who  had 
taken  video  training  from  a  commercial  source  rated  it  at  or 
above  80  percent  for  effectiveness. 

The  difference  in  average  ratings  for  commercial  and 
private  sources  of  CBT  training  increased  while  the 
difference  in  commercial  and  in-house  CBT  ratings  decreased 
from  question  1.  A  review  of  the  frequency  distribution  of 
responses  indicated  that  more  responses  occurred  at  levels 
below  70  percent  thus  lowering  the  average  rating  for  this 
method. 

Private  Training.  Ratings  for  effectiveness  of  private 
training  sources  increased  with  respect  to  value  ratings  for 
question  1  as  well  as  effectiveness  ratings  for  commercial 
and  in-house  sources.  Classroom  training  from  private 
sources  again  received  the  highest  average  rating  -  this 
time  for  effectiveness.  The  differences  between  this  source 
and  in-house  and  commercial  sources  of  video  and  CBT 
training  were  greater  for  question  2.  -  effectiveness,  than 
for  overall  value.  The  reader  should  note  these  differences 
in  light  of  the  fact  that  over  half  of  the  employees  who  had 
taken  two  or  more  sources  of  PC  training  still  rated 
private,  commercial,  and  in-house  equally.  Tests  for 
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statistical  significance  were  performed  to  identify  any  real 
differences  in  ratings. 

Statistical  Tests  for  Effectiveness  Ratings 

All  but  one  Z-test  was  inconclusive  at  the  five  percent 
level  of  significance.  The  mean  effectiveness  rating  for 
private  sources  of  classroom  training  was  statistically 
higher  than  the  mean  effectiveness  rating  for  in-house 
classroom  training.  One  other  Z  test  which  compared  private 
and  in-house  sources  of  CBT  training  was  significant  at  the 
six  percent  level;  but  this  conclusion  was  rejected. 

A  Mann-Whitney  test  to  compare  the  mean  ratings  for  in- 
house  and  commercial  classroom  training  was  inconclusive 
above  the  25  percent  level  of  probability.  The  results 
indicate  that  effectiveness  ratings  for  the  two  sources  of 
classroom  training  are  not  statistically  different. 

A  comparison  of  means  ratings  for  effectiveness  of  in- 
house  and  private  sources  of  classroom  indicated  a 
statistical  difference  at  approximately  the  61  percent  level 
of  probability,  which  suggests  the  two  can  be  considered 
equal  in  terms  of  effectiveness.  A  comparison  of  means  for 
vendor  and  private  source  classroom  training  effectiveness 
ratings  was  inconclusive  at  the  16  percent  level  of 
probability.  These  tests  suggest  that  all  three  sources 
may  considered  equal  in  terms  of  training  effectiveness, 
i.e.,  any  of  the  three  sources  provides  approximately  the 
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same  amount  of  material  which  can  be  mastered  per  training 
hour.  Results  of  the  tests  are  at  Appendix  D. 

Employee  Competency  Achieved  After  Training 

Survey  question  3.  was  used  to  measure  the  level  of 
competency  employees  felt  they  achieved  after  completing 
training  in  each  method/source  combination.  Respondents 
were  asked  to  assign  a  number  which  represented  the 
percentage  of  material  they  could  master  as  a  result  of  the 
type  of  training.  Table  IV.  contains  the  averages  of 
responses  for  each  method/source  combination. 

Primary  Conclusions .  Table  IV  also  provides 
conclusions  similar  to  those  for  Tables  II  and  III.  First, 
for  any  method,  there  are  no  differences  among  source. 
Second,  the  order  of  preference  for  methods  of  PC  training 
is  classroom,  then  CBT,  then  video  from  any  of  the  three 
sources.  The  reader  should  note  again  that  modal  responses 
may  be  higher  or  lower  than  average  ratings  indicated  in 
Table  IV. 
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Table 

IV.  Average 

Ratings  for  Competency 

Source 

Method 

Average  Rating 

In-House 

Classroom 

73.398 

In-House 

Video 

52.600 

In-House 

CBT 

62.798 

Contract 

Classroom 

74.028 

Contract 

Video 

53.077 

Contract 

CBT 

64.113 

Private 

Classroom 

77.035 

Private 

Video 

57.571 

Private 

CBT 

69.113 

In-House  Training.  The  differences  between  average 
ratings  for  employee  competency  after  training  from  in-house 
and  commercial  sources  were  less  than  differences  between 
the  two  sources  when  rated  for  effectiveness.  Ninety-six 
(96)  of  the  128  employees  who  had  taken  in-house  classroom 
training  rated  it  at  or  above  70  for  question  3.  The  modal 
response  for  in-house  classroom  was  80,  the  same  as  for 
question  1.  -  value,  and  question  2.  -  effectiveness.  One 
third,  or  27  of  the  81  people  who  had  taken  classroom 
training  from  both  private  and  in-house  sources  rated 
private  higher  than  in-house  for  competency  achieved  upon 
completing  training;  49  rated  the  two  equally.  Nearly  one 
third  (21)  of  the  69  employees  who  had  taken  classroom 
training  from  all  three  sources  rated  private  over  in-house. 

The  average  rating  for  competency  achieved  upon 
completion  of  in-house  video  training  was  the  lowest  for  all 
method/source  combinations.  The  modal  response  was  60,  the 


same  as  the  mode  for  effectiveness,  but  higher  than  the  mode 
of  50  for  overall  value.  Twenty- five  (25)  of  the  50 
employees  who  had  taken  video  training  rated  it  at  or  above 
60,  compared  to  18  of  the  50  who  rated  it  at  or  above  60  for 
value . 

None  of  the  33  employees  who  had  taken  CBT  training 
from  all  three  sources  rate  in-house  over  both  commercial 
and  private  sources;  2  rated  in-house  over  commercial  and  4 
rated  it  over  private  for  competency  achieved  after 
training.  Twenty-three  (23)  of  the  33  rated  in-house 
equally  with  the  other  two  sources  for  competency  achieved 
upon  completion  of  CBT  training. 

Commercial  Training.  Twenty-one  (21)  of  the  86  people 
who  had  taken  classroom  training  from  both  in-house  and 
commercial  sources  rated  commercial  higher  for  competency 
achieved  after  training.  Seventeen  (17)  of  the  69  who  had 
classroom  training  from  all  three  sources  rated  commercial 
over  in-house;  40  rated  the  two  equally.  Thirty-five  (35) 
rated  commercial  equally  with  private  sources  of  classroom 
training  for  competency  achieved. 

Twenty-seven  (27)  of  the  33  employees  who  had  taken 
video  training  from  all  three  sources  rated  commercial  and 
in-house  sources  the  same;  21  rated  commercial  and  private 
sources  the  same,  four  rated  commercial  over  in-house  or 
private . 
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Twenty-three  (23)  of  the  46  people  who  had  taken  CBT 
training  from  all  three  sources,  rated  commercial  equally 
with  in-house  and  private  sources;  two  rated  commercial 
higher  than  both  of  the  other  sources,  nine  rated  commercial 
over  in-house,  and  seven  rated  commercial  over  private. 

Private  Training .  Classroom  training  from  private 
sources  had  the  highest  average  rating  among  all  methods  and 
source  combinations  for  competency  achieved  after  completing 
training.  The  average  ratings  for  competency  achieved  from 
private  sources  were  higher  than  for  in-house  or  commercial 
sources  for  video  and  CBT  methods  as  well. 

Statistical  Tests  for  Competency  Ratings 

All  Z  tests  were  inconclusive  at  the  five  percent  level 
of  significance.  The  Mann-Whitney  tests  indicated  that  less 
than  a  20  percent  chance  existed  that  the  mean  rating  for 
commercial  classroom  training  was  higher  than  the  mean  for 
in-house  classroom  training  or  that  the  mean  for  private 
source  classroom  training  was  higher  than  that  for  in-house 
classroom  training.  This  indicates  that  all  three  sources 
of  training  lead  to  the  same  level  of  achieved  competency  as 
a  result  of  the  classroom  instruction  method,  approximately 
73  to  77  percent  of  what  employees  feel  the  ideal  course 
would  provide.  Results  of  the  tests  are  at  Appendix  E. 

The  next  section  provides  some  idea  of  what  one  hour. of 
classroom  training  from  these  two  sources  costs. 
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Costs  of  In-House  Classroom  Training 

The  author  had  to  rely  heavily  on  the  cooperation  of 
personnel  on  the  AFLC  headquarters  and  training  division 
staffs  for  cost  data.  Most  of  the  data  used  was  estimated 
by  the  staffs  based  on  current  and  historical  expenditure 
rates  and  planned  activities  for  the  remainder  of  FY89. 

Some  data  was  not  available;  most  notably  overhead  costs. 
Neither  staff  felt  they  could  accurately  estimate  management 
overhead  costs.  The  reason  cited  most  frequently  by  a  . 
number  of  knowledgeable  individuals  was  the  inability  of 
current  accounting  procedures  within  the  DOD  Resource 
Management  System  (RMS)  to  track  such  costs.  (1;  3;  4;  29) 

Sources  of  Cost  Data ♦  All  costs  except  personnel  costs 
were  provided  by  the  Staff  Development  Program  and  Policy 
Support  Branch  of  the  Directorate  of  Communication-Computer 
Systems,  Headquarters,  Air  Force  Logistics  Command,  Wright- 
Patterson  AFB  OH.  The  number  and  grades  of  civilian  and 
military  personnel  working  in  facility  110  were  provided. 
Building  costs  are  based  on  facility  records.  (3;  1;  5) 

Costs  for  personnel  were  provided  by  the  2750th  Air 
Base  Wing  Cost  Analysis  Branch,  Wright-Patterson  AFB  OH. 

The  branch  cited  Air  Force  Regulation  173-13,  Air  Force  Cost 
and  Planning  Factors  as  the  source  for  composite  wages  for 
military  personnel  and  acceleration  of  civilian  salaries  (to 
reflect  adjustments  for  retirement.  Social  Security, 
insurance,  etc.).  The  source  for  base  civilian  salaries 
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(before  acceleration)  was  cited  as  a  report  from  the  local 
civilian  personnel  office  called  the  General  Manager, 

General  Schedule,  and  Federal  Wage  System  Schedule  and  dated 
30  May  89.  (13) 

Costs  Included.  Appendix  F  contains  the  costs  and 
factors  used  to  calculate  the  cost  per  training  hour.  Costs 
are  divided  into  two  sections,  recurring  and  non-recurring. 
Non-recurring  costs  were  not  used  to  calculate  the  cost  per 
training  hour.  Recurring  costs  are  based  on  estimates  for 
expenditures  during  Fy89  for  the  following  categories: 
supplies,  travel,  facility  maintenance,  equipment 
maintenance,  replacement,  or  upgrade,  personnel,  and 
miscellaneous . 

The  majority  of  the  travel  expenditures  are  for  travel 
performed  by  a  mobile  training  team  (MTT)  which  is  composed 
of  five  rotating  instructors  from  the  division  staff  who 
travel  to  various  bases  within  AFLC.  Costs  for  supplies, 
equipment  maintenance,  and  facility  maintenance  for  training 
performed  by  the  MTT  at  other  bases  are  considered 
negligible . 

The  estimated  total  of  recurring  costs  for  the  AFLC 
Training  Division  for  FY89  is  $891,579.  An  estimated  4,777 
people  will  be  trained  in  facility  110  during  FY89.  Another 
approximately  4500  personnel  will  be  trained  by  the  MTT. 

The  average  number  of  hours  spent  in  PC  training  by  each 
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individual  student  is  16.  The  estimated  total  number  of  PC 
training  hours  for  FY89  is  148,432.  (3) 

Cost  per  Training  Hour.  The  estimated  annual  cost  per 
training  hour,  based  only  on  recurring  costs,  for  the  AFLC 
Training  Division  is  $6.01.  This  cost  is  based  on  the  total 
recurring  costs  estimated  for  FY89  divided  by  the  total 
number  of  student  training  hours  provided  by  the  division 
staff.  The  reader  is  cautioned  that  this  cost  does  not 
reflect  all  elements  necessary  to  be  considered  the  true 
cost  per  training  hour.  Since  this  cost  is  derived  using 
estimates  and  incomplete  cost  data  (e.g.  lack  of  overhead 
costs),  it  should  not  be  considered  absolutely  accurate. 

The  estimated  annual  value  of  the  7,451  square  feet 
utilized  by  the  training  division  in  building  110  is 
$147,468.  The  estimated  value  of  division  equipment  is 
$200,000.  The  training  staff  emphasizes  that  the  building 
and  equipment  could  be  utilized  by  virtually  any  other 
function  in  the  headquarters;  neither  are  unique  to  the 
training  facility's  mission.  Building  and  equipment  values 
are  adjusted  each  year  for  depreciation.  (1;  3) 

Mr.  Ron  Patton,  the  Chief  of  the  Training  Division 
located  in  building  110,  points  out  that  the  actual  recurring 
cost  per  training  hour  would  be  probably  lower  if  it 
reflected  time  spent  by  division  employees  in  support  of 
other  base  activities,  such  as  in  computer  consulting  and 
troubleshooting  for  other  organizations  on  base  (29) . 


Costs  of  Vendor- Provided  Classrooi  Training 

The  author  again,  as  with  AFLC,  had  to  rely  heavily  on 
the  cooperation  of  the  ASD  personnel  and  logistics  support 
staffs  as  sources  of  data.  Most  of  ASD's  data  was  also 
estimated  based  on  current  and  historical  expenditure  rates 
and  planned  activities  for  the  remainder  of  FY89.  ASD 
overhead  costs  were  also  not  available,  as  with  AFLC. 

Similar  reasons  were  cited  by  a  number  of  individuals  on  the 
ASD  personnel  and  logistics  support  staffs  for 
non-availability  of  overhead  costs,  namely  the  lack  of 
sufficient  detail  in  current  RMS  reports.  (17;  22) 

Sources  of  Cost  Data .  Mr.  Randy  Hopper  is  an  Employee 
Development  Specialist  assigned  to  the  ASD  Civilian 
Personnel  Office.  Mr.  Hopper  manages  the  three  contracts 
which  provide  microcomputer  training  to  ASD  employees.  He 
provided  the  contract  costs  per  hour,  the  number  and  grades 
of  personnel  assigned  to  his  function,  the  wages  for  the 
student  hire,  the  number  of  individuals  trained  by  contract, 
value  of  equipment  used,  the  total  square  feet  the  function 
utilizes  in  building  125,  the  facility  maintenance  costs  and 
the  average  number  of  hours  spent  in  training  by  each 
individual  (17).  Ms.  Debbie  Large,  a  supply  systems 
analyst,  provided  the  estimated  FY89  costs  for  supplies  for 
Mr.  Hopper's  function  (22).  Cost  per  square  foot  for  space 
utilized  by  Mr.  Hopper’s  function  were  obtained  from 
facility  records  (5) . 
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Costs  for  personnel  were  provided  by  the  2750th  Air 
Base  Wing  Cost  Analysis  Branch,  Wright-Patterson  AFB  OH. 

The  branch  cited  Air  Force  Regulation  173-13,  Air  Force  Cost 
and  Planning  Factors  as  the  source  for  composite  wages  for 
military  personnel  and  acceleration  of  civilian  salaries  (to 
reflect  adjustments  for  retirement.  Social  Security, 
insurance,  etc.).  The  source  for  base  civilian  salaries 
(before  acceleration)  was  cited  as  a  report  from  the  local 
civilian  personnel  office  called  the  General  Manager, 

General  Schedule,  and  Federal  Wage  System  Schedule  and  dated 
30  May  89.  (13) 

Costs  Included.  Appendix  G  contains  the  cost  data  and 
factors  used  to  calculate  the  cost  per  training  hour  for 
vendor-provided  PC  training.  The  costs  are  broken  into 
three  sections;  recurring  costs,  contract  costs,  and  non¬ 
recurring  costs.  Non-recurring  costs  were  not  used  to 
calculate  the  cost  per  training  hour.  Recurring  costs  are 
based  on  estimates  for  expenditures  during  FY89  for  the 
following  categories;  personnel,  supplies,  and  facility 
maintenance . 

Recurring  costs  do  not  include  any  official  funded 
travel  since  none  is  required  to  support  the  microcomputer 
training  contracts.  The  costs  also  do  not  reflect  any 
equipment  purchases  since  none  are  planned  for  the  next 
three  years.  The  recurring  costs  listed  at  Appendix  G  are 
those  required  to  support  Mr.  Hopper  and  his  staff  and  their 
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offices,  as  well  as  the  two  classrooms  located  in  building 
125  (17;  22) .  Approximately  25%  of  the  contracted  training 
is  conducted  by  vendor  employees  in  these  two  classrooms 
using  government  PCs.  Vendor  charges  are  the  same  for  both 
on-  and  off-base  training.  (17) 

The  total  estimated  recurring  costs  for  FY89  is 
$88,271.  The  estimated  number  of  employees  who  will  be 
trained  by  contract  is  4000  (17) .  The  average  number  of 
hours  spent  by  each  employee  in  training  is  18  (17) .  The 
total  estimated  number  of  PC  training  hours  per  year  is 
72000.  Total  recurring  costs  divided  by  total  training 
hours  results  in  a  recurring  cost  per  hour  of  $1.23.  This 
is  the  indirect  cost  of  providing  training  through  contract, 
and  should  be  added  to  the  average  contract  cost  per  hour. 

Contract  costs  are  based  on  the  average  of  the  costs 
per  day  stipulated  in  contract  agreements  between  ASD  and 
the  three  vendors.  Mr.  Hopper  estimates  that  the  total 
contracts  cost  for  FY89  microcomputer  training  will  be 
$350,000.  The  average  contract  cost  per  day  is  $25.  ASD 
employees  spend  an  average  of  six  hours  per  day  in  training 
which  results  in  an  average  contract  cost  per  training  hour 
of  $4.17.  (17) 

Cost  per  Training  Hour .  The  total  of  indirect  and 
direct  (contract)  costs  per  training  hour  is  $5.39.  The 
reader  is  cautioned  that  this  cost  may  not  reflect  all 
elements  necessary  to  be  considered  an  accurate  cost  per 
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training  hour.  Since  this  cost  is  derived  using  estimates 
and  incomplete  cost  data  it  should  not  be  considered 
completely  accurate. 

Non-recurring  costs  for  the  rooms  and  equipment  used  to 
support  Mr.  Hopper's  function  and  on-base  classroom  training 
total  to  $74,268.  Mr.  Hopper  indicated  that  the  floor  space 
and  equipment  could  be  used  by  almost  any  other  function; 
neither  are  unique  to  his  mission  (17) .  The  value  of  the 
facility  and  equipment  used  by  Mr.  Hopper's  function  is 
approximately  30  percent  of  the  value  of  the  facility  and 
equipment  used  to  support  the  AFLC  in-house  training 
function. 

Ranking  of  Methods  and  Sources 

Table  V.  indicates  the  ratios  of  benefits  to  cost-per- 
training  hour  for  in-house  and  commercial  sources  of  the 
classroom  method  of  microcomputer  training.  The  ratios  for 
each  source  are  computed  by  dividing  the  cost-per-training 
hour  by  the  average  rating  from  the  survey  for  each  benefit 
and  multiplying  by  1000.  These  ratios  indicate  the 
percentage  of  the  ideal  level  of  each  benefit  per  dollar 
spent.  The  ratios  do  not  reflect  the  fact  that  ratings  for 
the  two  are  not  necessarily  statistically  different;  this 
point  is  discussed  after  the  table  is  presented.  The  higher 
ratios  indicate  higher  ranking. 
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Table  V.  Cost-Benefit  Ratios  for  In-House 
and  Commercial  Classroom  Training 

In-House*  Commercial*  * 


Overall  Value 

80.80 

71.55 

Effectiveness 

86.77 

73.50 

Competency 

81.88 

72.81 

*  Cost-per-Training  Hour  is  $6.01 
**  Cost-per-Training  Hour  is  $5.39 

The  reader  may  interpret  the  ratios  as  follows:  In- 
house  classroom  training  provides  80.8  percent  of  the  ideal 
level  of  overall  value  to  the  employee’s  job  for  every 
dollar  spent  per  training  hour;  commercial  classroom 
training  provides  71.55  percent  of  the  ideal  level  of 
overall  value  for  each  dollar  spent  per  training  hour. 

The  reader  is  cautioned  that  the  ratios  in  Table  V. 
should  not  be  used  for  other  than  relative  comparison 
purposes.  Based  on  the  Mann-Whitney  tests,  ratings  for  both 
in-house  and  commercial  sources  of  classroom  training  are 
not  statistically  different  for  all  three  benefits.  Since 
no  statistical  difference  can  be  found  in  the  ratings  for 
both  sources,  cost-per-training  hour  may  be  used  as  the 
differentiating  factor.  The  reader  should  also  note  that 
some  evidence  is  presented  to  suggest  that  more  resources 
are  required  to  initially  establish  an  in-house  training 
function  than  to  establish  a  contract-based  training 
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program.  Cost  data  for  the  models  used  in  this  thesis 
suggest  that  the  difference  in  the  amount  of  resources 
required  can  be  as  much  as  300  percent. 

Based  on  the  cost  and  benefit  data  discussed  earlier  in 
this  chapter,  vendor-provided  training  provides  the  same 
levels  of  overall  value  to  the  employee,  training 
effectiveness,  and  employee  competency  upon  training 
completion  at  a  lower  cost. 

Table  VI.  contains  a  ranking  of  all  sources  and  methods 
based  on  the  average  of  survey  ratings  for  each  benefit; 
costs  are  not  considered. 

Table  VI.  Ranking  of  Sources  and  Methods 
of  Microcomputer  Training 


Source 

Method 

Value 

Effectiveness 

Competency 

In-House 

Classroom 

3 

3 

3 

Video 

9 

9 

9 

CBT 

6 

6 

6 

Commercial 

Classroom 

2 

2 

2 

Video 

8 

8 

8 

CBT 

5 

5 

5 

Private 

Classroom 

1 

1 

1 

Video 

7 

7 

7 

CBT 

4 

4 

4 

A  ranking  of  each  method  and  source  combination 
considering  all  three  benefits  simultaneously  would  produce 
the  same  order  iterated  in  Table  VI. 
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A  ranking  of  methods  without  consideration  of  source 
indicates  that  classroom  training  is  preferred  over  CBT 
training  which  is  preferred  over  video-based  PC  training. 

The  next  chapter  presents  conclusions  based  on  the 
findings  in  this  chapter  and  recommendations  for  further 
consideration  of  related  topics. 
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Introduction 


V.  Conclusions  and  Recomnendations 


Three  sources  of  microcomputer  training  and  three 
methods  used  by  these  sources  were  evaluated  in  this  thesis 
in  terms  of  their  overall  value,  effectiveness,  and  results. 
Examples  of  the  use  of  these  sources  and  methods  of  PC 
training  on  Wright-Patterson  AFB,  Ohio  were  also  discussed 
and  analyzed.  A  primary  focus  of  this  thesis  was  to  compare 
a  government  in-house  training  facility  to  a  government- 
contracted  vendor  training  program.  The  results  of  each  of 
the  evaluations,  analyses,  and  comparisons  yielded  the 
conclusions  and  recommendations  that  follow. 

Conclusions 

Based  on  the  cost  and  benefit  data  discussed  in  chapter 
IV.,  a  number  of  preliminary  conclusions  are  presented 
herein  for  the  reader's  careful  consideration.  The  word 
careful  is  stressed  since  the  benefit  data  is  based  on  one 
sample  of  a  large  population  and  the  cost  data  contains  a 
number  of  estimated,  rather  than  actual  costs. 

The  first  conclusion  that  can  be  made  from  analysis  of 
the  data  is  that  classroom  training  is  preferred  over  either 
CBT  or  video-based  PC  training.  The  mean  ratings  of 
classroom  training  for  all  three  benefits  were  statistically 
and  significantly  higher  than  ratings  for  the  other  two 
methods.  This  conclusion  echoes  the  popularity  of 
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instructor-led  training  as  the  traditional  and  most  often 
used  method  of  PC  training  as  mentioned  in  chapter  II. 

Another  conclusion  which  can  be  made  is  that  in-house, 
commercial,  and  private  sources  of  PC  training,  regardless 
of  the  methods  used,  are  equal  in  terms  of  overall  value  to 
government  employees,  training  effectiveness,  and  level  of 
competency  achieved  upon  completion.  Data  used  in  this 
thesis  provided  overwhelming  evidence  to  this  with  one  minor 
exception.  A  Z  test  to  compare  the  mean  ratings  for 
effectiveness  of  private  source  classroom  training  to  in- 
house  classroom  training  indicated  that  the  mean  for  private 
was  slightly  higher.  A  Mann-Whitney  test  for  the  same 
comparison  was  inconclusive.  Furthermore,  the  modal 
response  for  the  two  was  the  same.  An  admitted  weakness  in 
this  analysis  is  the  lack  of  adequate  data  on  the  viability 
of  utilizing  private  sources  for  training  government 
employees.  Despite  this  weakness,  the  reader  can 
effectively  conclude  that  all  three  sources  are  equally 
effect] ve . 

A  third  conclusion  which  can  be  made  is  that  little  or 
no  difference  exists  between  government  provided  in-house 
microcomputer  training  and  government  contracted  vendor- 
provided  training.  The  average  survey  ratings  for  the  two 
sources  were  not  statistically  different  and  the  cost-per- 
training  hour  for  each  source  differed  by  $.62  per  hour. 

This  suggests  that  decisions  about  which  source  to  choose 
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should  reflect  consideration  of  the  total  costs  of  choosing 
one  source  over  the  other.  As  mentioned  in  chapter  II, 
large  organizations  are  developing  in-house  training 
functions  when  they  find  the  cost  of  such  functions  is  lower 
than  the  cost  of  training  procured  by  contract  from  a 
commercial  vendor. 

The  last  conclusion  which  might  be  made  from  the  data 
presented  is  that  government  in-house  training  functions  can 
require  more  resources  than  training  through  contracted 
sources.  Although  no  specific  evidence  was  presented  to 
demonstrate  that  the  costs  of  each  source  of  PC  training  are 
representative  of  other  such  programs  elsewhere  in  the  Air 
Force  of  DOD,  some  general  points  can  be  made  nonetheless. 
Facility  and  equipment  costs  for  the  AFLC  in-house  training 
function  discussed  in  this  thesis,  for  example,  were  over 
300  percent  greater  than  those  for  the  ASD  training-by- 
contract  program.  The  amount  of  floor  space  and  equipment 
needed  for  in-house  training  functions  will  more  than  likely 
exceed  that  required  for  vendor-provided  training, 
particularly  if  most  or  all  vendor  training  is  conducted 
off-site.  Likewise,  less  personnel  are  needed  to  support  a 
training-by-contract  program  such  as  that  in  ASD. 

Decision  makers  should  consider  these  points  in 
addition  to  the  trade-offs  between  an  in-house  and  vendor 
training  programs.  Examples  of  such  trade-off 
considerations  are  initial  investment  of  resources. 
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recurring  costs,  flexibility  for  course  content,  length,  and 
focus,  and  employee  convenience.  These  considerations  along 
with  an  analysis  of  the  benefits  studied  herein,  i.e., 
overall  value,  training  effectiveness,  and  competency 
achieved,  should  form  the  basis  for  decisions  on  which 
sources  and  methods  of  PC  training  to  use. 

The  next  section  suggests  some  ideas  for  further  study 
of  these  and  related  topics  which  can  serve  to  improve  the 
information  available  to  managers  for  making  such  decisions. 

Recommended  Areas  for  Further  Study 

The  first  area  recommended  by  the  author  as  one  which 
requires  more  detailed  research  is  cost  data.  Admittedly, 
the  cost  data  used  herein  were  less  than  desirable  for 
making  concrete  conclusions  about  in-house  and  vendor- 
provided  training.  The  lack  of  accuracy  and  detail  in 
Resource  Management  Systems  reports  required  a  significant 
number  of  estimates  by  personnel  whose  jobs  did  not  require 
cost  estimation  expertise.  Efforts  to  collect  data  using 
cost  estimation  techniques  were  furthered  hampered  by  the 
lack  of  complete  historical  data  since  the  AFLC  and  ASD 
models  involved  major  changes  from  previous  years.  Data  for 
FY89,  if  collected  at  fiscal  year  end,  would  provide  a 
better  data  base  for  cost  analysis  since  both  programs  are 
expected  to  be  more  stable  during  this  fiscal  year. 
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A  second  area  which  requires  further  attention  involves 
competency  levels  achieved  upon  completion  of  PC  training. 
Currently,  AFLC  tests  all  students  at  the  end  of  each 
course.  ASD  does  not  require  the  three  contractors  to  do 
so,  and  no  plans  for  such  a  requirement  exist.  A  number  of 
private  sector  examples  should  be  available  in  the  near 
future,  based  on  indications  found  during  the  literature 
review  for  this  thesis.  Interviews  with  Air  Training 
Command  personnel  (not  cited)  also  indicated,  that  some  bases 
may  require  ve*- 'ors  to  test  employees  in  the  future;  as  of 
1989,  none  are  known  to  have  such  a  requirement. 

Another  area  which  should  be  researched  is  the 
viability  of  contracting  with  universities  and  colleges  to 
provide  PC  training.  This  was  briefly  mentioned  in  chapter 
II  as  being  attempted  on  a  limited  basis  by  some  bases  in 
the  Air  Force.  Based  on  the  survey  ratings  for  private 
sources,  this  alternative  might  prove  more  desirable  than 
other  sources  currently  used.  Cost  data  for  this  particular 
source  are  also  needed. 

Since  the  findings  in  this  thesis  are  based  on  only  one 
data  sample,  additional  samples  from  other  bases  and,  or 
services  should  be  surveyed  for  similar  data.  Such  surveys 
might  be  directed  toward  employees,  as  was  the  case  herein, 
or  toward  supervisors  of  the  employees. 

At  the  time  this  thesis  was  nearing  completion,  the 
popularity  of  CBT  and  interactive-video  instruction  was  on 
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the  rise.  Additional  research  about  these  two  methods  of  PC 
training  should  be  directed  at  their  potential  applications 
in  the  Air  Force  and  DOD. 

Sunmarv 

Computer  technology  and  its  rapid  development  have 
intensified  the  challenge  to  managers  of  all  organizations 
to  manage  computers  as  effectively  as  any  other  resource. 

The  Air  Force  has  recognized  that,  to  achieve  its  goal  of  a 
PC  on  every  desktop,  training  is  one  of  the  more  critical 
requirements  which  must  be  met.  Unfortunately,  meeting  the 
ever-changing  training  requirements  has  become  a  task  which 
requires  new  and  innovative  actions  by  base/wing  and  major 
command  level  managers. 

Some  of  the  methods  and  sources  of  PC  training 
available  to  Air  Force  managers  were  studied  in  this  thesis. 
A  primary  focus  was  the  comparison  of  the  two  more  popular 
sources  and  the  most  popular  method  of  microcomputer 
training  used  in  the  Air  Force  today:  in-house  and 
commercial  classroom  training.  The  results  of  this 
comparison  indicated  that  these  two  sources  of  classroom 
training  can  be  considered  equal  in  terms  of  overall  value 
♦■Q  the  employee's  job,  training  effectiveness,  and  the  level 
of  competency  achieved  by  the  employee  upon  training 
completion.  The  results  also  indicated  that  substantially 
more  resources  are  required  to  establish  and  maintain  in- 
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house  training  functions  than  to  establish  and  maintain 
support  vendor-provided  training.  Although  additional 
research  is  needed  to  refine  the  cost  estimates,  based  on 
the  data  used,  recurring  cost-per-training  hour  is  slightly 
higher  for  in-house  than  for  vendor-provided  training. 

A  survey  of  government  employees  revealed  that  they 
prefer  classroom  microcomputer  training  over  computer-based 
training  (CBT)  and  CBT  over  video-based  training  in  terms  of 
overall  value,  training  effectiveness,  and  competency 
achieved. 

These  findings  should  be  combined  with  further  research 
into  the  true  costs  and  benefits  of  in-house,  commercial, 
and  private  sources  for  a  variety  of  PC  training  methods  to 
include  computer-based  training  (CBT) ,  interactive-video 
instruction,  and  video-based  training.  Such  further 
research  should  be  aimed  at  providing  USAF  and  DOD  decision 
makers  with  quantitative  and  qualitative  data  for  use  in 
choosing  which  sources  and  methods  to  utilize  for  training 
government  employees. 
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Appendix  A:  Survey  Questionnaire  Used 


Survey  Control  No.  89-59 

SURVEY  EVALUATION  OF  MICROCOMPUTER  TRAINING 

This  survey  asks  you  to  evaluate  various  types  of  computer 
training  used  in  the  Air  Force  today.  In  the  questions 
below,  you  are  asked  to  use  numbers  which  represent  a 
percentage  of  the  ideal  level  of  employee  value,  training 
effectiveness,  and  student  competency  which  result  from  the 
type  of  training,  not  the  course  material.  Assume  that  100 
represents  the  ideal  level  and  use  numbers  in  increments  of 
10,  e.g.,  10,  20,  30,  40,  etc.,  up  to  100.  For  example,  if 
you  feel  video-based  training  provides  only  80%  of  the  what 
you'd  ideally  expect  to  get  from  a  course,  you  would  assign 
an  80  in  the  space  provided  to  the  left  of  that  type  of 
training . 

1.  This  question  is  designed  to  do  two  things:  a)  determine 
which  types  of  training  you  have  completed;  and  b)  how  you 
would  rate  the  types  of  training  you  have  received  in  terms 
of  overall  value  to  you  as  a  government  employee.  By 
overall  value,  we  mean  the  degree  to  which  the  training  made 
you  more  productive,  versatile,  and,  or  effective  in  your 
job.  Remember,  100  represents  the  ideal  level  of  value  to 
you  as  an  employee.  Assign  a  number  only  to  those  types  of 
training  you  have  completed  (even  if  you’ve  only  had  one). 

IN-HOUSE  (officially  provided  by  government  employees) : 

_  a.  Classroom  training  (traditional  student- 

instructor) 

_  b.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  c.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

COMMERCIAL  (through  use  of  government  contract) : 

_  d.  Classroom  training  (traditional 

student -instructor) 

_  e.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  f.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 
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PRIVATE  (through  your  own  efforts  or  off-duty) ; 

_  g.  Classroom  training  (high  school,  college, 

university) 

_  h.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape 
rather  than  a  human  instructor) 

_  i.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

2.  This  question  asks  you  to  determine  how  much  you  can 
effectively  learn  from  a  course  as  a  result  of  the  type  of 
training  used.  Assume  that  100  represents  the  most  material 
you  could  effectively  learn  in  one  hour  of  training.  For 
each  of  the  types  of  training  you  have  taken,  assign  a 
number  which  represents  the  amount  of  material  you  feel  can 
be  learned  for  each  hour  of  training. 

IN-HOUSE  (officially  provided  by  government  employees) : 

_  a.  Classroom  training  (traditional  student- 

instructor) 

_  b.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  c.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

COMMERCIAL  (through  use  of  government  contract) : 

_  d.  Classroom  training  (traditional 

student- instructor) 

_  e.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  f.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

PRIVATE  (through  your  own  efforts  or  off-duty) : 

_  g.  Classroom  training  (high  school,  college, 

university) 

_  h.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape 
rather  than  a  human  instructor) 

_  i.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

3.  This  question  asks  you  to  determine  your  level  of 
competence  upon  completion  of  each  type  of  training  you 
received.  For  example,  if  you  took  a  course,  but  felt  you 
could  only  master  60%  of  the  course  material 

because  of  the  type  of  training  used,  assign  a  60. 
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IN-HOUSE  (officially  provided  by  government  employees) 

_  a.  Classroom  training  (traditional  student- 

instructor) 

_  b.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  c.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

COMMERCIAL  (through  use  of  government  contract) : 

_  d.  Classroom  training  (traditional 

student-instructor) 

_  e.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape  rather  than  a 
human  instructor) 

_  f.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

PRIVATE  (through  your  own  efforts  or  off-duty) : 

_  g.  Classroom  training  (high  school,  college, 

university) 

_  h.  Video-based  training  (typically  requires 

students  to  observe  a  video  tape 
rather  than  a  human  instructor) 

_  i.  Computer-based  training  (computer  software 

"instructs"  and  prompts  student  for  input) 

Thank-you  very  much  for  your  time.  We  appreciate  you 
cooperation.  Please  use  the  enclosed  return  envelope  to 
return  this  survey. 
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Appendix  B :  Electronic  Spreadsheet  Printouts 
Containing  Data  Collected 


The  printouts  follow  this  page 
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Question  1.  Overall  Value 


Di-KVSE 

OMl'L 

PRIVATE 

Response 

Qass 

Viiteo 

CBT 

Class 

Video 

CBT 

Class 

Video 

CBT 

1 

60 

10 

20 

50 

10 

20 

50 

10 

10 

2 

50 

50 

50 

50 

50 

50 

3 

80 

70 

4 

30 

70 

40 

70 

40 

80 

5 

60 

70 

90 

70 

80 

100 

50 

80 

100 

6 

80 

7 

100 

90 

70 

100 

90 

70 

100 

90 

70 

8 

80 

9 

20 

80 

20 

80 

10 

90 

70 

90 

70 

90 

60 

11 

70 

50 

60 

80 

50 

70 

40 

40 

60 

12 

100 

13 

80 

80 

80 

80 

80 

80 

70 

80 

80 

14 

60 

40 

10 

70 

40 

20 

20 

20 

60 

15 

60 

80 

16 

60 

80 

17 

60 

60 

18 

50 

50 

70 

60 

50 

80 

60 

50 

80 

19 

50 

50 

50 

80 

50 

50 

20 

20 

20 

20 

20 

50 

90 

50 

80 

21 

100 

40 

70 

100 

60 

70 

70 

60 

70 

22 

80 

70 

23 

90 

24 

90 

80 

90 

80 

100 

80 

25 

70 

50 

26 

50 

50 

27 

70 

100 

28 

50 

80 

50 

29 

80 

10 

10 

40 

10 

10 

80 

10 

10 

30 

100 

100 

100 

31 

90 

32 

90 

60 

30 

90 

60 

30 

80 

50 

20 

33 

70 

34 

80 

60 

70 

80 

60 

70 

80 

60 

70 

35 

ICO 

36 

70 

40 

60 

80 

40 

60 

37 

50 

80 

38 

60 

39 

70 

40 

50 

41 

50 

42 

80 

70 

90 

90 

90 

43 

80 

44 

90 

50 

80 

90 

50 

80 

100 

50 

70 

80 


QyflHnn  1.  Overall  Value 


iN-aousE 


OJfl’L 


PraVATE 


Response 


Haf”  Video  CBT  Class  Video  CBT  Class  Video  OT 


45 

80 

46 

70 

47 

80 

48 

80 

49 

70 

50 

75 

10 

51 

95 

52 

60 

53 

90 

54 

90 

80 

55 

80 

80 

56 

85 

57 

65 

58 

80 

59 

80 

40 

60 

10 

61 

70 

62 

90 

63 

70 

70 

64 

100 

65 

66 

90 

70 

67 

75 

68 

70 

50 

69 

100 

70 

50 

71 

72 

90 

73 

90 

74 

100 

75 

100 

76 

90 

75 

77 

90 

50 

78 

80 

79 

80 

80 

50 

81 

70 

20 

82 

80 

83 

70 

84 

10 

10 

85 

100 

86 

80 

30 

87 

90 

88 

10 

90 

70  80  70 

25 

90  80 

10 

90  90  80 

40  100  85 

85 

100 

60  80  40 

40 

90 

80  80  70 

100  100 

20 

80  80  70 

75 
70 

50  50 

70 

80 


60  90 

80  90  50 

90  80 

50  100 

30  70  20 

80  80 

10 

50  80  50 

90 


60 

80 

85 

75 

95 

50 

30 

10 

90 

85 

80 

45 

100 

60 

85 

70 

100 

60 

90 

90 

90 

80 

90 

90 

100 

100 

80 

60 

50 

90 

80 

90 

50 

90 

50 

25 

30 

80 

20 

80 

80 

70 

70 

70 

70 

50 

85 


81 


8  88  8  o  8888  S8  S  88 


Question  1.  Overall  Value 


HHEUSE 

aitf'L 

PRIVATE 

Response 

Class 

Video 

CBT 

Qass 

Video 

®r 

Class 

Video 

CBT 

89 

90 

30 

90 

50 

50 

91 

70 

92 

100 

60 

80 

100 

60 

80 

100 

60 

90 

93 

65 

85 

75 

90 

77 

80 

94 

80 

90 

90 

100 

95 

80 

50 

70 

% 

90 

50 

97 

80 

40 

30 

90 

40 

30 

90 

40 

30 

98 

30 

35 

45 

30 

25 

85 

99 

60 

30 

10 

80 

10 

10 

100 

80 

80 

80 

101 

90 

70 

90 

80 

102 

90 

70 

80 

70 

90 

70 

70 

103 

80 

100 

90 

50 

100 

104 

80 

60 

60 

105 

50 

40 

95 

106 

70 

50 

40 

80 

90 

107 

40 

80 

80 

108 

80 

30 

30 

80 

30 

30 

109 

70 

80 

80 

110 

90 

90 

111 

70 

70 

70 

112 

100 

50 

80 

100 

30 

60 

90 

40 

50 

U3 

60 

30 

30 

70 

70 

114 

20 

40 

60 

70 

60 

70 

60 

70 

60 

115 

30 

70 

50 

116 

90 

80 

100 

90 

80 

100 

90 

80 

100 

117 

90 

80 

100 

90 

U8 

90 

U9 

100 

70 

70 

100 

70 

70 

100 

70 

70 

120 

100 

90 

100 

90 

90 

80 

121 

60 

50 

60 

122 

70 

80 

50 

123 

60 

10 

30 

60 

20 

40 

70 

30 

124 

20 

90 

125 

90 

70 

90 

70 

90 

126 

80 

70 

70 

80 

60 

50 

127 

80 

60 

128 

60 

70 

60 

90 

129 

100 

50 

100 

50 

100 

50 

130 

60 

70 

70 

80 

131 

40 

20 

50 

132 

75 

30 

50 

80 

90 

82 


Question  1.  Overall  Value 


Response 

m-aausE 

GCMl'L 

PRIVAH: 

Qass 

Video 

CBT 

Class 

Video 

CBT 

Qass 

Video 

CBT 

133 

100 

65 

100 

95 

70 

65 

134 

80 

70 

60 

80 

70 

60 

80 

70 

60 

135 

90 

90 

136 

100 

10 

40 

100 

10 

40 

100 

10 

40 

137 

80 

70 

80 

90 

70 

80 

80 

70 

80 

138 

90 

70 

90 

139 

80 

60 

80 

60 

90 

140 

60 

70 

141 

60 

142 

70 

80 

60 

143 

40 

20 

144 

60 

80 

145 

80 

70 

80 

60 

146 

60 

147 

10 

148 

75 

15 

10 

50 

40 

10 

50 

40 

10 

149 

90 

70 

70 

90 

70 

70 

90 

70 

75 

150 

60 

20 

40 

60 

20 

50 

10 

20 

30 

151 

70 

10 

20 

60 

10 

40 

100 

10 

10 

I  Resp's 

128 

50 

84 

108 

39 

62 

86 

42 

71 

Average:  74.375 

46.700 

59.643 

75.324 

50.641 

62.742 

76.419 

51.429 

63.028 

Std  Dev. 

20.66 

23.12 

24.15 

18.69 

22.93 

24.41 

20.89 

24.38 

25.94 

FRBQUE2CY  DIsnUBUTIGN  -  minber  of  responses  within  each  increment  of  10. 


Qass 

Video 

(BT 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

10 

3 

7 

5 

1 

4 

4 

1 

5 

6 

20 

3 

4 

4 

2 

3 

2 

2 

4 

2 

30 

3 

4 

7 

0 

3 

4 

2 

2 

6 

40 

2 

7 

6 

3 

5 

> 

2 

4 

1 

50 

6 

10 

10 

12 

6 

8 

7 

6 

8 

60 

14 

3 

10 

8 

5 

6 

8 

5 

8 

70 

20 

9 

18 

14 

7 

12 

9 

8 

9 

80 

34 

5 

11 

33 

4 

12 

15 

6 

19 

90 

26 

1 

9 

22 

2 

7 

25 

2 

5 

100 

17 

0 

4 

13 

0 

4 

15 

0 

7 

128 

50 

84 

108 

39 

62 

86 

42 

71 

83 


VSTtl^ 


Qass 

Video 

CBT 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

40 

10 

10 

40 

10 

10 

30 

10 

10 

50 

50 

50 

50 

50 

50 

60 

60 

10 

60 

10 

60 

10 

70 

40 

60 

80 

50 

60 

80 

60 

80 

100 

70 

90 

100 

100 

90 

100 

100 

90 

100 

100 

80 

30 

70 

30 

70 

70 

30 

80 

30 

80 

30 

30 

60 

80 

40 

60 

80 

60 

60 

80 

80 

80 

80 

80 

80 

80 

80 

80 

80 

80 

60 

40 

30 

60 

30 

30 

50 

40 

40 

50 

80 

50 

80 

70 

80 

50 

50 

80 

60 

50 

80 

60 

60 

80 

80 

50 

50 

50 

50 

50 

50 

50 

50 

40 

60 

100 

80 

100 

100 

60 

80 

100 

60 

80 

100 

60 

80 

90 

90 

80 

60 

80 

70 

80 

80 

80 

60 

60 

80 

50 

90 

90 

40 

50 

80 

80 

10 

10 

70 

10 

10 

80 

10 

10 

100 

80 

100 

100 

80 

40 

10 

80 

40 

10 

70 

30 

10 

90 

90 

60 

70 

90 

60 

70 

90 

60 

70 

90 

40 

40 

60 

50 

40 

60 

90 

80 

60 

80 

50 

50 

80 

60 

80 

90 

90 

80 

80 

50 

70 

90 

50 

70 

100 

50 

70 

84 


DHCUSE 


Question  2.  Effectiveness 
acm'L 


espouse 


PRIVATE 


Class 

Video 

CBT 

Qass 

Video 

CBT 

Qass 

Videc 

45 

70 

46 

80 

47 

70 

60 

48 

95 

80 

50 

49 

75 

65 

80 

75 

75 

80 

75 

50 

45 

10 

25 

51 

80 

90 

60 

80 

50 

52 

90 

75 

53 

80 

20 

20 

54 

90 

80 

85 

90 

80 

85 

80 

70 

55 

90 

50 

50 

100 

60 

55 

100 

40 

56 

85 

57 

40 

60 

60 

50 

58 

75 

80 

80 

59 

90 

50 

60 

90 

50 

60 

60 

10 

90 

61 

70 

60 

80 

62 

80 

80 

90 

63 

70 

70 

80 

90 

80 

90 

90 

100 

64 

100 

80 

100 

80 

100 

65 

100 

66 

80 

50 

60 

70 

50 

50 

50 

40 

67 

30 

30 

40 

68 

100 

70 

100 

69 

100 

70 

60 

60 

40 

71 

70 

72 

90 

73 

20 

10 

74 

90 

75 

90 

76 

75 

60 

70 

75 

77 

50 

30 

40 

50 

30 

40 

50 

40 

78 

80 

79 

25 

75 

80 

90 

80 

85 

85 

85 

85 

75 

81 

80 

20 

30 

80 

20 

30 

80 

20 

82 

70 

70 

70 

70 

70 

83 

80 

80 

84 

70 

10 

10 

85 

5 

86 

40 

40 

30 

70 

60 

90 

70 

40 

87 

90 

60 

88 

80 

75 

80 


70 


80 


85 


S  S  SSS8  ^  S8» 


Questicn  2.  Effectiveness 


D^HOUSE 

GOM'L 

PRIVATE 

Response 

Qass 

Video 

CBT 

Qass 

Video 

(BT 

Qass 

Video 

CBT 

89 

100 

50 

90 

70 

70 

91 

30 

92 

90 

80 

100 

90 

80 

100 

90 

80 

100 

93 

65 

75 

80 

85 

78 

80 

94 

70 

90 

80 

100 

95 

90 

70 

70 

96 

90 

50 

97 

80 

40 

30 

90 

40 

30 

90 

40 

30 

98 

15 

25 

40 

20 

20 

55 

99 

100 

60 

60 

100 

60 

60 

100 

70 

70 

70 

101 

80 

80 

80 

80 

102 

90 

90 

70 

90 

80 

90 

90 

103 

20 

40 

30 

10 

40 

104 

80 

60 

60 

105 

75 

75 

90 

106 

70 

50 

40 

80 

90 

107 

20 

80 

80 

108 

80 

30 

30 

80 

30 

30 

109 

90 

90 

90 

110 

90 

90 

111 

100 

100 

100 

112 

60 

50 

70 

40 

40 

40 

100 

80 

90 

113 

80 

20 

30 

80 

80 

114 

50 

60 

70 

40 

30 

50 

60 

60 

60 

115 

100 

90 

100 

116 

50 

60 

80 

60 

60 

85 

65 

60 

85 

117 

100 

100 

100 

100 

118 

90 

119 

100 

80 

80 

100 

80 

80 

100 

80 

80 

120 

80 

70 

80 

70 

70 

60 

121 

60 

50 

60 

122 

50 

80 

70 

123 

30 

10 

60 

40 

10 

50 

50 

50 

124 

30 

90 

125 

80 

20 

80 

20 

80 

126 

70 

40 

40 

80 

60 

60 

127 

90 

80 

128 

90 

70 

80 

90 

129 

100 

10 

100 

50 

100 

50 

130 

40 

80 

30 

80 

131 

20 

30 

50 

132 

80 

50 

70 

95 

95 

86 


Questico  2.  Effectiveness 


Hi-HDUSE 

OMI’L 

PRIVAIE 

NcSpOiBW 

— 

'  ■  - 

■ -  — 

— 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

133 

100 

65 

100 

95 

70 

65 

134 

80 

70 

60 

80 

70 

60 

80 

70 

60 

135 

90 

90 

136 

100 

10 

40 

100 

10 

40 

100 

10 

40 

137 

80 

60 

70 

90 

70 

90 

80 

60 

80 

138 

100 

100 

100 

D9 

60 

70 

60 

5 

60 

140 

90 

90 

141 

90 

142 

80 

80 

80 

143 

40 

40 

144 

60 

80 

145 

80 

60 

80 

60 

146 

30 

147 

10 

148 

75 

20 

5 

50 

40 

10 

50 

40 

10 

149 

90 

70 

70 

90 

70 

70 

90 

70 

70 

150 

80 

20 

60 

80 

20 

60 

60 

40 

60 

151 

30 

10 

60 

70 

10 

20 

30 

10 

60 

1  Resp's 

128 

50 

84 

108 

39 

62 

86 

42 

71 

Average:  69.258 

46.500 

59.048 

73.333 

49.103 

59.919 

75.558 

55.476 

66.408 

Std  Dev. 

24.18 

22.68 

24.67 

20.95 

23.75 

26.05 

20.39 

23.60 

23.35 

ERBQUEmCY  DISHUEUTION  -  nuDix  r  of  responses  within  each  increnent  of  10. 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

10 

4 

7 

6 

2 

5 

5 

2 

4 

4 

20 

4 

4 

3 

0 

3 

4 

0 

1 

0 

30 

8 

4 

9 

4 

3 

4 

2 

2 

5 

40 

8 

6 

7 

7 

5 

3 

1 

8 

4 

50 

9 

9 

4 

10 

5 

7 

10 

3 

6 

60 

9 

10 

13 

8 

8 

9 

9 

9 

9 

TO 

13 

5 

16 

12 

3 

6 

6 

4 

10 

80 

35 

4 

15 

30 

6 

12 

25 

8 

19 

90 

25 

0 

7 

20 

0 

9 

15 

1 

7 

100 

13 

1 

4 

15 

1 

3 

16 

2 

7 

128 

50 

84 

108 

39 

62 

86 

42 

71 

87 


Question  3.  Gonijetency 

DHCUSE  Qatl'L  PRIYKn; 

Response  =  —  -  ■■ -=  -  - 


Class 

Video 

CBT 

Class 

Video 

CBT 

Class 

Video 

CBT 

1 

70 

30 

40 

60 

30 

30 

50 

20 

20 

2 

50 

50 

50 

50 

50 

50 

3 

70 

80 

4 

90 

100 

90 

90 

90 

100 

5 

50 

70 

100 

50 

70 

100 

50 

70 

100 

6 

70 

7 

100 

90 

80 

100 

90 

80 

100 

90 

80 

8 

90 

9 

60 

40 

60 

40 

10 

100 

100 

100 

100 

100 

100 

11 

70 

60 

80 

70 

60 

80 

70 

60 

80 

12 

90 

13 

90 

90 

90 

90 

90 

90 

90 

90 

90 

14 

80 

50 

70 

80 

60 

60 

80 

60 

60 

15 

40 

80 

16 

40 

80 

17 

40 

30 

18 

50 

50 

70 

60 

50 

80 

70 

50 

80 

19 

60 

60 

60 

50 

50 

50 

80 

80 

80 

20 

10 

40 

80 

40 

100 

21 

100 

100 

100 

100 

lOO 

100 

100 

100 

100 

22 

80 

70 

23 

80 

24 

80 

70 

80 

70 

90 

70 

25 

80 

40 

26 

80 

50 

27 

70 

90 

28 

50 

90 

50 

29 

80 

50 

10 

80 

50 

40 

80 

50 

40 

30 

50 

50 

50 

31 

70 

32 

100 

80 

60 

100 

80 

60 

80 

60 

50 

33 

90 

34 

90 

70 

80 

90 

70 

80 

90 

80 

60 

35 

90 

36 

70 

60 

70 

70 

60 

70 

37 

80 

100 

38 

60 

39 

50 

40 

50 

41 

50 

42 

90 

90 

90 

90 

90 

43 

80 

44 

70 

50 

80 

100 

50 

70 

80 

50 

70 

88 


Question  3.  OooiKtency 


IN-HOUSE 

ocm'L 

PRIVATE 

Response 

Class 

Video 

(BT 

Class 

Video 

CBT 

Class 

Video 

CBT 

45 

90 

46 

70 

80 

47 

70 

40 

48 

95 

70 

90 

49 

70 

75 

75 

70 

75 

80 

70 

85 

50 

100 

100 

100 

100 

51 

70 

80 

50 

70 

50 

52 

80 

90 

85 

53 

90 

5 

5 

54 

70 

65 

75 

70 

75 

75 

80 

80 

85 

55 

95 

60 

60 

80 

60 

35 

100 

60 

30 

56 

70 

57 

45 

60 

60 

55 

58 

75 

80 

85 

59 

70 

30 

50 

70 

30 

50 

60 

10 

90 

61 

50 

40 

70 

62 

80 

80 

80 

63 

60 

50 

70 

80 

80 

80 

90 

90 

90 

64 

80 

85 

80 

85 

80 

85 

65 

60 

80 

66 

80 

50 

60 

70 

50 

50 

50 

40 

50 

67 

60 

60 

70 

68 

90 

70 

90 

69 

90 

70 

70 

60 

50 

80 

71 

70 

72 

70 

70 

73 

50 

40 

74 

100 

70 

75 

100 

70 

76 

80 

50 

60 

80 

77 

70 

40 

40 

70 

40 

40 

70 

40 

40 

78 

80 

79 

90 

90 

90 

90 

80 

85 

85 

85 

85 

50 

85 

81 

70 

15 

10 

70 

15 

10 

75 

30 

15 

82 

70 

70 

70 

70 

70 

70 

83 

80 

70 

84 

70 

10 

10 

85 

95 

86 

80 

70 

80 

90 

70 

90 

80 

60 

80 

87 

90 

60 

88 

75 

70 

89 


Class 

Video 

or 

Video 

CBT 

Class 

Video 

CBT 

89 

90 

90 

90 

60 

60 

91 

20 

92 

80 

60 

100 

80 

60 

100 

80 

60 

100 

93 

80 

80 

75 

85 

80 

82 

94 

100 

80 

60 

90 

95 

80 

70 

70 

96 

90 

75 

97 

60 

50 

20 

70 

50 

30 

70 

50 

30 

98 

15 

15 

20 

15 

20 

40 

99 

100 

60 

60 

100 

70 

70 

100 

70 

70 

70 

101 

70 

80 

90 

70 

102 

50 

60 

50 

60 

60 

60 

60 

103 

80 

100 

90 

40 

90 

104 

80 

60 

60 

105 

40 

40 

75 

106 

60 

40 

30 

70 

80 

107 

10 

50 

75 

108 

70 

40 

40 

80 

40 

40 

109 

100 

100 

100 

110 

90 

90 

m 

100 

100 

100 

112 

100 

40 

80 

100 

40 

80 

100 

80 

90 

113 

90 

60 

30 

70 

80 

114 

50 

60 

40 

70 

50 

60 

60 

70 

80 

115 

100 

90 

100 

116 

75 

50 

100 

85 

50 

100 

85 

50 

100 

117 

100 

100 

100 

100 

118 

80 

119 

100 

80 

80 

100 

80 

80 

100 

80 

80 

120 

80 

70 

80 

70 

70 

70 

121 

40 

30 

30 

122 

30 

70 

60 

123 

70 

30 

90 

70 

30 

90 

70 

90 

124 

60 

90 

125 

100 

60 

100 

60 

100 

126 

80 

60 

60 

80 

60 

40 

127 

70 

50 

128 

100 

80 

90 

90 

129 

50 

50 

75 

50 

75 

50 

130 

80 

90 

90 

90 

131 

40 

30 

50 

132 

75 

30 

70 

80 

90 

90 


Question  3.  Conyetency 


IN-HOUSE 

GCm'L 

PRIVA'IE 

weojjtjiiac 

- Lua - 

- u  — 

Class 

Video 

CBT 

Class 

Video 

CBT 

Qass 

Video 

CBT 

133 

90 

55 

90 

85 

63 

55 

134 

80 

80 

70 

80 

80 

70 

80 

80 

70 

135 

90 

90 

136 

70 

30 

40 

70 

30 

40 

70 

30 

40 

D7 

90 

60 

80 

90 

60 

80 

80 

60 

80 

138 

80 

100 

60 

139 

80 

50 

80 

50 

80 

140 

70 

80 

141 

90 

142 

80 

90 

80 

143 

40 

40 

144 

60 

60 

145 

80 

50 

80 

40 

146 

50 

147 

10 

148 

75 

20 

5 

50 

40 

10 

50 

40 

10 

149 

95 

5 

85 

95 

5 

85 

95 

5 

85 

150 

80 

10 

20 

10 

10 

30 

10 

20 

30 

151 

30 

10 

60 

30 

10 

60 

70 

10 

20 

i  Resp's 

128 

50 

84 

108 

39 

62 

86 

42 

71 

Average: 

73.398 

52.600 

62.798 

74.028 

53.077 

64.113 

77.035 

57.571 

69.113 

Std  Dev. 

20.88 

23.69 

25.32 

19.20 

23.44 

23.84 

17.90 

21.65 

23.11 

EREQUEMCY  DISTnCEEUTICN  -  nunber  of  responses  within  each  increnent  of  10. 

Qass 

Video 

or 

Qass 

Video 

CBT 

Qass 

Video 

CBT 

10 

3 

4 

6 

1 

3 

3 

1 

2 

1 

20 

2 

3 

2 

1 

2 

1 

0 

2 

3 

30 

2 

5 

3 

3 

4 

3 

1 

2 

3 

40 

6 

4 

10 

4 

3 

6 

3 

4 

5 

50 

10 

9 

8 

10 

8 

8 

6 

7 

7 

60 

9 

11 

11 

7 

7 

9 

5 

10 

6 

70 

23 

6 

11 

21 

4 

7 

15 

5 

11 

80 

33 

3 

16 

27 

5 

10 

23 

6 

12 

90 

21 

3 

8 

20 

2 

10 

18 

3 

14 

100 

19 

2 

9 

14 

1 

5 

14 

1 

9 

128 

50 

84 

108 

39 

62 

86 

42 

71 
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Appendix  C:  Results  of  Statistical  Tests 
for  Survey  Ratings  on  Overall  Value 


Statistical  tests  were  performed  to  test  the  hypothesis  that 
means  for  ratings  of  each  of  two  sources  compared  were 
statistically  different.  The  levels  of  probability 
indicated  in  the  respective  columns  below  are  the  levels  at 
which  the  hypothesis  can  be  rejected.  Z-test  results  are 
based  on  two-tailed  tests.  Results  for  Mann-Whitney  tests 
are  based  on  two-tailed  tests  for  comparison  of  means  of 
non-paired  sample  data.  A  five  percent  level  of 
significance  for  the  two-tailed  tests  was  required  for  a  95 
percent  level  of  probability  to  reject  the  hypothesis  that 
the  means  are  equal.  That  is,  a  probability  in  the  table 
below  that  is  less  than  .95  indicates  no  significant 
difference  between  means.  Abbreviations  are  used  for 
sources:  I  for  in-house,  C  for  commercial,  and  P  for 
private. 


Sources 

Method 

Compared 

Z-test 

Mann-Whitnev 

Classroom 

C-I 

.2886 

.1114 

P-I 

.5160 

.8836 

P-C 

.2886 

.8836 

Video-based 

C-I 

.5762 

P-I 

.6528 

P-C 

.1114 

CBT 

C-I 

.5528 

P-I 

.5934 

P-C 

.0478 
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Appendix  D:  Results  of  Statistical  Tests 
for  Survey  Ratings  on  Effectiveness 


Statistical  tests  were  performed  to  test  the  hypothesis  that 
means  for  ratings  of  each  of  two  sources  compared  were 
statistically  different.  The  levels  of  probability 
indicated  in  the  respective  columns  below  are  the  levels  at 
which  the  hypothesis  can  be  rejected.  Z-test  results  are 
based  on  two-tailed  tests.  Results  for  Mann-Whitney  tests 
are  based  on  two-tailed  tests  for  comparison  of  means  of 
non-paired  sample  data.  A  five  percent  level  of 
significance  for  the  two-tailed  test  was  required  for  a  95 
percent  level  of  probability  to  reject  the  hypothesis  that 
the  means  are  equal.  That  is,  a  probability  in  the  table 
below  that  is  less  than  .95  indicates  no  significant 
difference  between  means.  Abbreviations  are  used  for  ■ 
sources:  I  for  in-house,  C  for  commercial,  and  P  for 
private. 


Sources 


Method 

Compared 

Z-test 

Mann-Whitnev 

Classroom 

C-I 

.8324 

.2510 

P-I 

.9596 

.6156 

P-C 

.5408 

.1664 

Video-based 

C-I 

.3970 

P-I 

.9342 

P-C 

.7738 

CBT 

C-I 

.1586 

P-I 

.9426 

P-C 

.8664 

Appendix  E:  Results  of  Statistical  Tests 
for  Survey  Ratings  on  Coppetencv  Achieved 

Statistical  tests  were  performed  to  test  the  hypothesis  that 
means  for  ratings  of  each  of  two  sources  compared  were 
statistically  different.  The  levels  of  probability 
indicated  in  the  respective  columns  below  are  the  levels  at 
which  the  hypothesis  can  be  rejected.  Z-test  results  are 
based  on  two-tailed  tests.  Results  for  Mann-Whitney  tests 
are  based  on  two-tailed  tests  for  comparison  of  means  of 
non-paired  sample  data.  A  five  percent  level  of 
significance  for  the  two-tailed  test  was  required  for  a  95 
percent  level  of  probability  to  reject  the  hypothesis  that 
the  means  are  equal.  That  is,  a  probability  in  the  table 
that  is  less  than  .95  indicates  no  significant  difference 
between  means.  Abbreviations  are  used  for  sources:  I  for 
in-house,  C  for  commercial,  and  P  for  private^ 


Sources 

Method 

Compared 

Z-test 

Mann-Whitnev 

Classroom 

C-I 

.1896 

.1896 

P-I 

.8262 

.1974 

P-C 

.7372 

.0008 

Video-based 

C-I 

.0718 

P-I 

.7062 

P-C 

.6266 

CBT 

C-I 

.2510 

P-I 

.8948 

P-C 

.7776 
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Appendix  F;  Estinated  AFLC  Training  Costs  for  FY89 


(estimated) 

FY  89  {$) 
$93,100 
$46,550 
$728,829 
$22,500 
$600 
$29,260 


$891,579 


NOK-RBOJRRING  OOCTS  -  (for  BUB  110) 

BuHding  $147,468 

BEjuipmait  $200,000 


$347,468 


Ntnber  of  People  Trained 
In  BUK  110:  ATT! 

By  MIT:  4500 


Total;  9277 

Avg  hrs  per  individual 
in  Training:  16 

Estimated  I  of  PC 
Training  hours/Yr:  148432 


Est  cost  per  hour:  $6.01 


No.  trained  in  BLDG  110:  4777 
Avg  hrs  per  individual:  16 
Est  i  of  training  hrs/yr:  76432 

Est  cost  per  hour:  $4.55 


RBOURRINS  OBTS  - 

Supplies 

Travel 

Personnel 

Equipment 

Misc 

Facility  Naint 


Grade 

No. 

Ann  Wage 

Total 

GM13 

2 

$63,320 

$126,639 

GS12 

1 

$53,239 

$53,239 

GSll 

2 

$44,432 

$88,864 

GS9 

2 

$36,710 

$73,421 

G87 

5 

$30,032 

$150,160 

GS5 

7 

$24,251 

$169,757 

E6 

1 

$30,555 

$30,555 

E5 

1 

$36,194 

$36,194 

Total: 

21 

$728,829 
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(estimated) 

Fre9  ($) 
$85,271 
$2,500 
$500 


$88,271 


ocffn^cr  ccer  pek  hour 
Avg  csost  per  day:  $25.00 

Training  hours  per  day:  6 


Avg  cost  per  hour:  $4.17 


NDK-RBCURRING  CDBTS  -  (for  BUK  125) 

Building  $48,268 

Cost  per  Sq  Ft:  $14.06 
I  of  aq  ft:  3433 

Equipment  $26,000 


$74,268 


Personnel 


Gcade 

No. 

Ann  Wage 

Total 

GS12 

1 

$53,239 

$53,239 

GS6 

1 

$30,032 

$30,032 

Stud  Teop 

1 

$2,000 

$2,000 

Total: 

2 

$85,271 

Est  I  liained:  4000 

Average  f  hours  spent 
in  Training:  18 


Estimated  I  of  PC 
Training  hours/Yr:  72000 


Est  cost  per  hour:  $1.23 
Avg  contract  $/hr:  $4.17 


Total:  $5.39 


RECURRING  OOGTS  - 

Personnel 
Siqiplies 
Facility  Haint 
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- -  The  purpose  of  this  study  was  to  evaluate  the 

methods  and  sources  of  microcomputer  training  currently  used 
in  the  Air  Force  in  terms  of  their  overall  value  to  the  job, 
training  effectiveness,  and  level  of  competency  achieved 
upon  training  completion.  The  three  methods  studied  were 
classroom  instruction,  video-based  training,  and  computer^ 
based  training  (CBT) .  The  three  sources  studied  were  in- 
house  training  functionn,  government-contracted  training, 
and  private  (e.g.,  colleges  and  universities).  One  hundred 
and  flfty-ono.  government  employees  rated  each  method  and 
source  of  PC  training  they  had  experienced. 

^^lanriroom  training  was  found  to  be  the  most  often  used 
and  the  most  popular  method  of  PC  training,  regardless  of 
source.  The  preference  of  delivery  method  was  classroom, 

CBT,  then  video-based  training,  regardless  of  source. 

A  cost-benefit  analysis  was  performed  to  compare 
two  most  often  used  sources  of  classroom  training:  in-house 
•  uitt  government -contractor.  Statistical  results  indicated 
that  the  two  source's  are  equal  in  terms  of  the  benefits 
stated  abtjvo;  cost-per-training  hour  was  slightly  lower  for 
vendor-provided  training  than  for  in-house  training. 

Resources  required  to  establish  and  maintain  in-house 
training  functions  were  substantially  greater  than. .those 
required  to  support  vendor-provided  training  programs. 

The  study  found  that  college  or  university  PC  training 
was  rated  equally  with  in-house  and  commercial  sources  of 
classroom,  CBT,  and  video-based  training.  One  particular 
recommendation,  in  light  of  the  challenge  presented  to 
managers  for  improving  microcomputer  training  in  the 
workplace,  is  to  investigate  the  possibility  of  training 
government  employees  by  contract  with  local  educational 
instltutionr. .  Potential  benefits  include  better  trained  and 
eihicar.od  employees,  higher  morale,  and  savings  of  operations' 
■rad  maintenance  Cundr?. 


